AD 


Contract  Number  DAMD17-94-C-4127 


TITLE:  Real  Time  3D  Ultrasound  for  Physiological  Monitoring 


PRINCIPAL  INVESTIGATOR: 


Richard  J.  Littlefield 


CONTRACTING  ORGANIZATION: 


Battelle  Pacific  Northwest  Laboratory 
Richland,  Washington  99352 


REPORT  DATE : 


September  1997 


TYPE  OF  REPORT:  Annual 


PREPARED  FOR: 


U.S.  Army  Medical  Research  and  Materiel  Command 
Fort  Detrick,  Maryland  21702-5012 


DISTRIBUTION  STATEMENT: 


Approved  for  public  release;  - 
distribution  unlimited 


The  views,  opinions'  and/or  findings  contained  in  this  reoort  are 
ose  o  t  e  author (s)  and  should  not  be  construed  as  an  official 
Department  of  the  Army  position,  policy  or  decision  unless  so 
designated  by  other  documentation.  unless  so 


unc  quality  rKsrBCTBD  a 


19971208  013 


REPORT  DOCUMENTATION  PAGE 


OMB  No.  0704-0188 


K.  TITLE  AND  SUBTITLE 


casQund 'fm-  P)iyc;-i 


report  TYPE  AND  DATES  COVERED  '  •  . 

.Annual  (15  Sep  96  -14  Sep  97) 

■ - - -  I  5.  FUNDING  NUMBERS 


NSN  7640-01-280-5500 


Standard  Form  298  (Rev.  2-89) 
298.?02^  by  ANSI  Scd.  Z39.ia 


FOREWORD 


Opinions,  interpretations,  conclusions  and 

those  of  the  author  and  are  not  necessarily  endorsed  by  the  U.S. 
Army . 

Where  copyrighted  material  is  quoted,  permission  has  been 
obtained,  to  use  such  material . 

f^<[L  Where  material  from  documents  designated  for  limited 
distribution  is  quoted,  permission  has  been  obtained  to  use  the 

material . 

IZ-(L  citations  of  commercial  organizations  and  trade  names  in 
report  do  not  constitute  an  official  Department  of  Amy 
Ldorsement  or  approval  of  the  products  or  services  of  these 
organizations. 

///^  In  conducting  research  using  animals, 

af^he^red  to  the  "Guide  for  the  Care  and  Use  of  Laboratory 
J^imals  "  prepared  by  the  Committee  on  Care  and  use  of  Laboratory 

Animals 'of  the  Institute  of 

Research  Council  (NIH  Publication  No.  86-23,  Revised  IPao) 

lifr  For  the  protection  of  human 
SdhSred  to  policies  of  applicable  Federal  Law  45  GFR  46 . 

In  conducting  research  utilizing  recombinant  DNA  technolo^, 
^inveI??ga?or(s)  adhered  to  current  guidelines  promulgated  by 
the  National  Institutes  of  Health. 

lit  In  the  conduct  of  research  utilizing  recombinant  DNA,  the 
^sJ?gSor(s)  adhered  to  the  NIH  Guidelines  for  Research 
Involving  Recombinant  DNA  Molecules. 

t( T-  +-V.  nf  research  involving  hazardous  organisms, 

l^inveSgrtSr  “a^frertf  the  CDC-NXH  Guide  for  Biosafety  in 

Microbiological  and  Biomedical  Laboratories. 


TABLE  OF  CONTENTS 


1.  Introduction . . . . . ...1 

2.  Experimental  Methods  and  Results . . . 7 

3.  Conclusions . . . . . . . ..9 

Appendices 


1  “MUSTPAC-1:  3-D  Ultrasound  Telemedicine  Tool  for 

Deployment  Situations  in  Bosnia  and  the  European 
Theater.  Final  Report,  ‘Bosnia  Task’,  PNL  IRB  #  94-6-1, 
Project  #22258”  . . . . . . . . . . . . ....A1-1 

2  Patent  Application:  “Ultrasound  Telemedicine  System  with 

Virtual  Reality” . . . A2-1 

3  Discover  Award  -  “Computer  Hardware  and  Electronics: 

Pacific  Northwest  National  Laboratory’s  Portable 
Ultrasound  Unit”,  Discover  Magazine,  July  1997 . A3-1 

4  “Ultrasound  marches  to  the  front”,  Portable  Design  Magazine 

Cover  article,  June  1997 . . . A4-1 

5  “Backpack  remote  medicine  proves  its  worth”,  Janes 

International  Defense  Review,  February  1997 


A5-1 


6  Presentation  to  MUSTPAC-2  Expert  Review  Panel,  August  5, 


1997... . A6-1 

7  MUSTPAC-2  Prototype,  10/15/97 . A7-1 

8  Computational  Focusing  -  Support  for  Off-Axis  Spherical 

Illumination  with  2-D  Array  Sensors . ....A8-1 


1.  INTRODUCTION 

Inexpensive,  portable  diagnostic  imaging  systems  can  play  a  key  role  in  decreasing 
battlefield  fatalities  and  reducing  the  cost  of  military  health  care.  Ultrasound  imaging  is  a 
particularly  promising  modality  because  it  does  not  use  ionizing  radiation  (unlike  X- 
rays),  does  not  require  large  heavy  equipment  (unlike  magnetic  resonance  imaging),  and 
has  been  shown  through  long  use  to  be  safe  and  effective  when  used  by  highly  trained 
practitioners. 

However,  in  current  practice,  ultrasound  is  basically  an  online  two-dimensional  (2-D) 
scanning  procedure  that  produces  a  sequence  of  images  xmder  interactive  hands-on 
control  by  the  diagnostician.  Each  image  represents  a  slice,  through  the  body  at  the 
corresponding  ultrasound  probe  position.  These  images  typically  are  difficult  to  interpret, 
requiring  a  trained  ultrasonographer  with  years  of  experience  to  make  more  than  the 
simplest  diagnoses,  This  need  for  an  expert  interpreter  makes  it  attractive  to  use 
ultrasound  in  a  telemedicine  setting,  sending  images  from  the  patient’s  location  to  a 
skilled  diagnostician  somewhere  else.  However,  with  conventional  2-D  ultrasound, 
considerable  skill  is  required  even  to  position  the  sensor  probe,  since  this  must  be  done 
interactively  as  diagnosis  progresses.  Thus  it  is  problematic  to  use  conventional  2-D 
ultrasound  in  a  telemedicine  setting,  due  to  the  need  for  a  highly  skilled  operator  to  scan 
the  patient.. 

Three-dimensional  (3-D)  ultrasound  imaging  offers  the  potential  to  overcome  these 
difficulties,  thus  providing  a  diagnostically  valuable,  low-cost,  real-time  imaging 
modality  suitable  for  operation  and  use  under  emergency  conditions  by  non-specialists. 
Because  3-D  volumes  show  more  context  than  2-D  slices,  it  becomes  easier  for  users  to 
understand  spatial  relationships  and  detect  abnormal  conditions.  Positioning  the  sensor 
so  as  to  acquire  useful  images  is  also  easier  with  3-D,  because  volumetric  data  can  readily 
be  rotated  and  realigned  to  good  viewing  positions,  largely  independent  of  the  original 
•  sensor  position.  This  potentially  allows  useful  3-D  ultrasoimd  data  to  be  taken  by  an 
inexperienced  operator,  then  transmitted  to  and  interpreted  by  a  remote  expert. 
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While  3-D  ultrasound  imaging  has  been  investigated  periodically  for  over  20  years,  its 
adoption  into  routine  use  has  been  hindered  by  clumsy  equipment,  long  image  acquisition 
times,  and  the  difficulties  of  visualizing  the  3-D  clouds  of  relatively  noisy  data  produced 
by  speckle  and  directional  effects  of  ultrasound.  However,  recent  advances  in  transducer 
array  technology,  computational  hardware  speed,  and  improved  image  reconstruction  and 
visualization  methods  now  appear  sufficient  to  permit  these  obstacles  to  be  overcome. 

These  considerations  prompted  Battelle  to  submit  to  DARPA,  in  response  to  solicitation 
BAA94-14  in  early  1994,  a  proposal  titled  “Real-Time  High  Resolution  3-D  Ultrasonic 
Imaging  for  Physiological  Monitoring” .  This  proposal  laid  out  the  vision  of  a  three  stage 
effort,  roughly  8  years  in  length,  leading  to  the  development  of  an  imaging  “  bed” , 
roughly  10,000  square  centimeters  in  size,  containing  an  array  of  high  resolution 
ultrasonic  transducers  and  providing  real-time  3-D  visualization  of  many  physiological 
and  anatomical  structures. 

The  first  stage  of  this  vision,  and  the  focus  of  the  proposal,  was  a  3 -year  project  to 
develop  a  field  prototype  Advanced  IMaging  System  (AIMS).  This  prototype  would 
consist  of  a  lightweight,  portable  ultrasonic  imaging  system  envisioned  as  containing  a  5 
cm  by  5  cm  two-dimensional  transducer  array,  computer  hardware  and  software  for  real¬ 
time  3-D  holographic  image  reconstruction  and  visualization,  and  a  stereo  vision  headset 
for  3-D  image  display.  The  system  was  envisioned  as  being  used  to  rapidly  detect  foreign 
objects  and  bleeding  in  the  body  cavity,  lungs,  or  extremities. 

The  Battelle  proposal  to  develop  an  AIMS  prototype  was  accepted  by  DARPA,  and  the 
project  began  in  September  1994  with  two  major  components: 

•  Research  and  develop  advanced  sensor  technology,  in  particular,  2-D  transducer 
arrays  utilizing  computational  holographic  focusing  to  acquire  3-D  images  in  real 
time. 

•  Research  and  develop  one  or  more  fully  functional  prototype  systems,  suitable  for 
clinical  and/or  field  use,  to  investigate  and  demonstrate  the  utility  of  3-D  ultrasound 
as  a  medical  imaging  tool  for  use  by  non-specialist  operators. 
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In  early  stages  of  the  projects,  it  was  planned  that  these  two  components  would  proceed 
sequentially,  with  the  prototype  system(s)  being  based  on  newly  developed  2-D  array 
transducers  and  thus  appearing  late  in  the  project. 

However,  as  the  project  progressed,  it  became  apparent  that  a  more  productive  strategy 
was  to  pursue  both  components  in  parallel,  with  prototype  systems  being  based  on 
currently  available  1-D  array  transducer  technology.  This  strategy  was  adopted  in  FY95 
and  has  proved  very  effective. 

In  FY95,  results  in  each  research  and  development  component  were  as  follows: 

•  Sensor  technology:  Laboratory  research  using  mechanically  scanned  simulations  of  2- 
D  transducer  arrays  confirmed  that  high  quality  images,  fully  focused  everywhere  in 
the  3-D  field  of  view,  could  be  obtained  by  using  computational  holographic  focusing 
techniques  in  conjvmction  .with  large  arrays  (128x128)  of  high-frequency  (5  MHz) 
transducers.  However,  the  supporting  electronics  and  computational  requirements  to 
use  such  large  arrays  appeared  beyond  the  reach  of  current  technology.  These 
requirements  could  be  met  with  smaller,  lower  firequency  arrays,  such  as  32x32  at 
IMHz.  Tests  at  1  MHz  did  not  produce  encouraging  results  when  used  to  image 
commercially  available  plastic  ultrasound  phantoms  of  solid  organs  (liver  and  breast). 
However,  the  phantoms  were  not  specifically  designed  for  use  at  such  low 
frequencies,  so  it  was  not  clear  whether  the  results  accurately  reflected  the  medical 
utility  of  1  MHz  ultrasound.  Accordingly,  it  was  decided  to  perform  further  testing  at 
1  MHz  in  FY96,  using  a  different  type  of  phantom  and  focusing  on  diagnosis  of 
abdominal  blood  pooling. 

•  Prototype  systems:  a  clinically  usable  3-D  ultrasound  system  based  on  “  sequential  B- 
scan”  technology  (mechanical  sweep  of  a  conventional  ultrasound  probe)  was , 
developed.  This  system  was  displayed  at  the  October  1995  annual  meeting  of  the 
AUSA  (Association  of  the  U.S.Army)  in  Washington  DC,  where  it  was  favorably 
reviewed  by  many  Army  persormel.  More  importantly,  the  system  was  placed  into  use 
in  the  clinic  of  Dr. Christian  Macedonia  at  Madigan  Army  Medical  Center  in  Tacoma, 
Washington,  for  an  extended  evaluation  to  occur  in  FY96. 
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In  FY96,  research  and  development  continued  in  both  of  these  areas,  but  with  a  further 
shift  of  emphasis  toward  the  development  of  field-usable  systems. 

•  The  most  important  product  from  FY96  was  a  second-generation  prototype  system, 
called  MUSTP AC- 1  (Medical  [or  Military]  UltraSound,  Three-dimensional  and 
Portable,  with  Advanced  Communications),  that  provides  powerful  telemedicine 
capabilities.  Delivery  and  field  demonstration  of  the  MUSTP  AC-1  was  accomplished 
during  July-September  1996.  On  July  8,  Battelle  delivered  a  MUSTP  AC-1  system  to 
the  U.S.  Army  at  Ft.  Detrick.  Following  compatibility  testing  under  supervision  of 
the  Center  for  Total  Access  (CTA,  Ft.Gordonj  Georgia),  the.  MUSTP  AC-1  was 
shipped  to  Landstuhl,  Germany,  for  further  evaluation.  On  August  7,  it  was  deployed 
to  the  212th  Mobile  Army  Surgical  Hospital  in  Tuzla,  Bosnia,  while  a  second 
MUSTP  AC- 1  remained  at  LRMC  (Landstuhl  Regional  Medical  Center)  to  serve  as  a 
receiving  station.  Additional  secondary  receiving  stations  were  established  and  used 
at  Madigan  Army  Medical  Center  (Tacoma,  WA,  USA),  Fraunhofer  Center  for 
Research  in  Computer  Graphics  (Providence,  RI,  USA),  and  Georgetown  University 
Medical  Center  (Washington,  DC,  USA).  The  MUSTP  AC- 1  remained  in  Bosnia 
xmtil  it  was  redeployed  to  Georgetown  on  Sept.8. 

As  a  result  of  the  success  of  MUSTP  AC- 1,  a  change  within  scope  was  negotiated  in  late 
FY96  that,  adjusted  the  project’s,  priorities  to  reduce  the  level  of  effort  on  2-D  array 
development,  emphasizing  instead  further  development  of  field-usable  systems  based  on 
sequential  B-scan  technology. 

In  FY97,  notable  accomplishments  under  this  contract  were: 

1 .  further  demonstration  and  publication  of  the  prototype  MUSTP  AC- 1  telemedicine 
system,  including  acceptance  of  one  national  award  for  Technology  Innovation; 

2.  patent  application  for  the  MUSTP  AC- 1  architecture  and  system  concept; 

3.  technical  development  of  a  third-geiieration  prototype  MUSTP  AC-2  system 
(currently  estimated  50%  complete,  with  a  working  prototype  available); 

4.  laying  groundwork  for  technology  transfer  of  MUSTP  AC-2;  and 
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5.  continued  research  on  sensor  technology. 

Further  details  are  provided  as  follows: 

1.  Further  demonstration  and  publication  of  MUSTPAC-1. 

On  May  31,  1997,  largely  as  a  result  of  the  Germany/Bosnia  field  demonstration, 
MUSTPAC-1  won  the  Discover  Award  for  Technology  Innovation  in  Computer 
Hardware  and  Electronics  (Discover  Magazine,  Jul  97). 

Demonstrations  and  discussions  about  MUSTPAC  were  invited  and  given _ mostly 

with  funding  from  other  organizations  —  at  the  AUSA  Annual  Meeting  (Washington, 
DC,  14-16  Oct  96),  the  AUSA  Telemedicine  Conference  (Tysons  Comer,  VA,  4-6 
Mar  97),  the  Military  Medical  Capabilities  Conference  (Knoxville,  TN,  30  May  97), 
Grand  Rounds  at  Walter  Reed  Army  Medical  Center  and  Georgetown  University 
Medical  Center  (Feb  97  and  May  97),  the  Spanish  Society  of  OB/GYN  meeting 
(Spain,  Jun  1997),  the  Society  of  Minimally  Invasive  Therapeutics  Meeting  (Japan,  Jul 
97),  the  MMVR-5  conference  (San  Diego,  CA,  Jan  97),  NASA  Johnson  Space  Center 
(Houston,  TX,  Sep  96  and  Jan  97),  the  Tribal  Healthcare  2000  Conference  (San 
Diego,  CA,  15-17  Jul  97),  and  the  AMEDD  Center  and  School  Conference  on  Force 
Stmcture  and  Requirements  in  Telemedicine  (Jul  1997). 

The  MUSTPAC-1  has  received  a  high  level  of  attention  and  support  from  the  public 
news  media.  Technical  articles  about  MUSTPAC,  written  by  magazine  staff, 
appeared  in  Jane’s  International  Defense  Review  (Feb  97,  pg.l5)  and  Portable  Design' 
Magazine  (cover  article,  Jun  97).  A  non-technical  article,  associated  with  the 
Discover  Award,  appeared  in  Discover  Magazine  (Jul  1997,  pp.74-75).  These  articles 
are  provided  as  Appendices  3, 4,  and  5  of  this  report.  In  addition  many  non-technical 
articles  appeared  in  AP  newspapers  and  on  broadcast  television,  including  CBS, 
CNBC,  and  The  Discover  Channel. 

2.  Patent  application. 

On  June  25, 1997,  a  patent  application  titled."  Ultrasound  Telemedicine  System  with 
Virtual  Reality” ,  was  filed  in  support  of  the  U.S.  Department  of  Defense,  covering 
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the  system  concept  and  certain  details  of  the  MUSTP  AC- 1  system.  A  copy  of  this 
patent  application  is  provided  as  Appendix  2. 

3.  Technical  development  of  MUSTPAC-2. 

Technical  development  of  the  MUSTPAC-2  successor  system  has  made  good 
progress.  This  effort  is  based  on  MUSTP  AC-1  experiences  and  has  the  goal  of 
developing  an  FDA-approved  manufacturable  system  with  improved  operating 
characteristics.  At  present  (October  1997),  the  research  and  development  effort  is 
approximately  50%  complete.  A  working  prototype  has  been  assembled  and  is  being 
evaluated.  Further  development  and  documentation  activities  are  imderway  to  make 
the  unit  manufacturable  and  suitable  for  FD  A  5 10(k)  certification.  Meetings  of  the 
MUSTPAC-2  Expert  Panel  were  held  on  Feb.25  and  Aug.5,  1997,  to  discuss  system 
development  status  and  solicit  guidance  on  all  aspects  of  system  development  and 
future  use.  A  set  of  high  level  briefing  slides  fi:om  the  Aug.5  meeting  is  provided  as 
Appendix  6.  A  technical  overview  of  the  MUSTPAC-2  and  its  development  process 
is  provided  in  the  Section  2  of  this  report. 

4.  Technical  transfer  of  MUSTPAC-2. 

Technical  transfer  of  the  MUSTP  AC  technology  is  an  ongoing  activity  that  combines 
.  1)  documenting,'  demonstrating,  and  discussing  MUSTP  AC  with  potential  partners 
and  users  of  the  technology,  and  2)  establishing  partnerships  with  one  or  more 
.  industrial  companies  to  provide  manufacturing  and  continued  support  of  MUSTP  AC 
units. 

Business  discussions  were  held  with  4  established  ultrasound  companies,  leading  to 
one  agreement-in-principle  for  manufacture  of  MUSTPAC  systems. 

5.  Development  of  sensor  technology. 

An  improved  method  was  developed  for  computational  holographic  reconstruction  of 
data  obtained  firom  a  2-D  array  sensor  using  off-axis  illumination. 

Additional  details  on  MUSTPAC-2  and  sensor  technology  are  provided  in  the  following 
section. 
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2.  EXPERIMENTAL  METHODS  AND  RESULTS 

Technical  activities  in  FY97  (October  1996  through  September  1997)  fell  primarily  in 
two  areas: 

•  Sensor  technology.  An  improved  method  was  developed  for  computational 
holographic  reconstruction  of  data  obtained  from  a  2-D  array  sensor  using  off-axis 
illumination.  This  method  allows  off-axis  spherical-wave  illumination  to  be  used,  an 
extension  over  earlier  techniques  which  supported  only  plane- wave  illumination.  In 
addition,  a  simulation  study  was  performed  to  evaluate  the  Lockheed-Martin  “BUDI” 
system.  It  was  found  that  the  BUDI  system  appears  compatible  vdth  the  new 
reconstruction  technique.  Key  viewgraphs  illustrating  these  results  are  presented  in 
Appendix  8.  No  laboratory  experimental  work  on  sensor  technology  was  performed 
in  FY97,  and  the  simulation  studies  comprised  a  relatively  small  part  of  the  project 
(approximately  3%  of  expenditures). 

•  Development  of  the  MUSTPAC-2  ultrasound  telemedicine  system  as  a  successor  to 
the  highly  successful  MUSTPAC-1.  This  effort  was  the  focus  of  the  project  in  FY97, 
comprising  over  90%  of  total  expenditures.  The  development  effort  and  results  are 
described  in  more  detail  in  the  remainder  of  this  section. 

Appendices  1  and  2  describe  the  MUSTPAC-1  in  considerable  detail,  and  Appendix  7 
shows  an  overview  of  the  MUSTPAC-2. 

In  brief,  MUSTPAC-2  retains  the  high  level  architecture  of  the  MUSTPAC-1  and  most  of 
its  operating  characteristics,  but  MUSTPAC-2  has  been  re-engineered  for  improved  size, 
weight,  robustness,  and  usability.  Almost  every  major  subsystem  has  been  replaced.  In 
addition,  the  re-engineering  process  is  being  performed  in  compliance  with  FDA  510(k) 
requirements. 

Specific  technical  changes  incorporated  in  the  MUSTPAC-^2  include: 

•  Redesigned. the  3-D  Paddle  electromechanical  scanner  to  provide  a  fully  sealed 
operating  mechanism  capable  of  inunersion  cleaning  in  standard  disinfectant 
solutions.  This  was  accomplished  through  the  use  of  magnetic  coupling  of  physical 
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force  from  the  internal  drive  mechanism,  through  an  impermeable  aluminum  shell,  to 
an  external  probe  carrier. 

•  Replaced  the  Silicon  Graphics  computer,  display,  and  keyboard  with  a  Pentium  laptop 
(Toshiba  Tecra  740CDT)  augmented  with  a  separate  video  capture  card  (Osprey  100). 

•  Converted  all  software  to  run  under  the  Solaris  operating  system.  Solaris  is  an 
interim  step,  chosen  to  produce  an  operational  prototype  MUSTPAC-2  on  the  shortest 
possible  schedule.  The  operating  system  for  the  final  (September  1 998)  version  of 
MUSTPAC-2  will  be  Windows/NT. 

•  Extended  the  TeleInViVo  visualization  software  to  provide  additional  features 
requested  by  MUSTPAC-1  users. 

•  Upgraded  fi'om  Immersion  Probe™  to  Immersion  Corporation’s  newer  EndoArm 
version  of  their  Micro  Scribe  product,  for  use  as  the  MUSTPAC’s  Virtual  Ultrasound 
Probe  user  interface  device. 

•  Changed  packaging  from  backpack  concept  to  hard  case  with  wheels. 

Although  these  changes  are  conceptually  straightforward,  some  of  them  proved 
surprisingly  difficult  to  accomplish.  Video  capture  was  an  important  example.  Based  on  . 
discussions  with  vendors  in  1995  and  1996,  MUSTPAC  development  staff  believed  that 
adequate  video  capture  capability  (640x480  pixels  at  15  fps  or  better)  would  be 
conveniently  available  in  Pentium  laptops,  either  built-in  or  as  a  CardBus  module. 
However,  it  turned  out  that  neither  capability  was  developed  by  industry  according  to  the 
projected  schedule,  and  after  several  false  starts,  the  MUSTPAC  project  was  eventually 
forced  to  use  a  PCI-bus  video  capture  card.  This  in  turn  required  adding  an  external  PCI- 
bus  capability  to  MUSTPAC-2,  with  attendant  increase  in  size,  weight,  and  complexity  of 
packaging.  In  addition,  a  significant  amount  of  programmer  effort  was  required  to  adapt 
vendor-supplied  video  capture  device  driver  software  to  the  special  needs  of  the 
MUSTPAC  project. 

In  retrospect,  it  can  be  seen  that  rapid  development  of  the  high  quality  MUSTPAC-1 
prototype  was  in  large  part  due  to  the  availability  and  selection  of  a  particularly  well 
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suited  computer  platform.  The  Silicon  Graphics  Indy  computer  used  in  the  MUST? AC- 1 
provided  an  integrated  system,  fully  supported  by  a  single  vendor,  encompassing  all  of  ■ 
the  required  capabilities  in  video  capture,  display,  computational  speed,  and 
communication  software.  It  was  a  noticeable  setback  to  the  project  that  Silicon  Graphics 
chose  to  discontinue  the  Indy,  replacing  it  only  with  systems  that  were  too  large  and 
heavy  to  be  viable  for  continued  MUSTP AC  development. 

As  of  October  1997,  however,  the  difficulties  of  platform  conversion  have  been  overcome 
and  a  prototype  MUSTP  AC-2  system  is  operational.  Further  evaluation  and  development 
are  planned  for  the  next  year,  leading  to  completion  of  the  MUSTP  AC-2  final  version  in 

September  1998. 

3.  CONCLUSIONS 

Success  of  the  MUSTPAC-1  under  field  conditions  in  FY  96  provided  considerable 
validation  that  3-D  ultrasound  can  be  an  effective  telemedicine  tool,  allowing  a  non¬ 
specialist  operator  in  the  field  to  obtain  high  quality  scans  that  can  be  interpreted  by  a 
remote  expert. 

In  fact,  development  and  field-testing  of  the  MUSTPAC-1  amounted  to  early  delivery  of 
one  major  goal  of  the  project;  a  prototype  portable  3-D  ultrasound  system  usable  under 
field  conditions  by  a  non-specialist 

Much  further  development  remains,  however,  to  refine  the  MUSTP  AC  technology  into  a 
system  suitable  for  routine  use  imder  a  wide  range  of  conditions.  Good  progress  was 
made  in  this  direction  during  FY97,  with  a  prototype  MUSTP  AC -2  becoming  functional 
in  early  October  1997.  Development  of  the  MUSTP  AC-2  will  continue  in  FY98,  with 
the  scheduled  completion  of  an  FDA-approved  system  by  the  September  1998. 
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ABSTRACT 

Ultrasound  is  a  popular  technology  for  medical 
imaging  of  soft  tissues  because  it  is  inexpensive,  safe, 
and  relatively  portable.  However,  the  use  of  ultrasound 
imaging  in  remote  areas  has  been  limited  because 
conventional  real-time  2-D  (two-dimensional)  ultrasound 
imaging  requires  that  a  highly  skilled  operator,  capable  of 
diagnostic  decisions,  be  present  at  the  patient's  location 
to  point  the  data  acquisition  probe.  3-D  ultrasound  data 
acquisition  has  the  potential  to  remove  this  limitation,  by 
allowing  an  operator  with  no  diagnostic  skills  to  collect 
high  quality  scans  that  can  be  interpreted  by  a  remote 
expert.  This  capability  is  illustrated  by  the  MUSTPAC-1, 
a  portable  3-D  ultrasound  telemedicine  system  recently 
developed  for  the  U.S.  military.  In  August  1996,  the 
MUSTPAC-1  system  was  field-tested  by  the  U.S.  Army 
in  Gerrriany  and  Bosnia.  This  report  discusses  design 
and  implementation  of  the  MUSTPAC-1  system  and 
summarizes  results  of  the  Bosnia  field  test. 

INTRODUCTION 

Ultrasound  is  a  commonly  used  method  for 
medical  imaging  of  soft  tissues.  It  is  safe,  inexpensive, 
and  quick. 

However,  conventional  2-D  (two-dimensional) 
ultrasound  imaging  has  the  significant  drawback  of 
requiring  a  highly  skilled  operator  to  be  physically 
present  at  the  patient’s  location.  This  is  because 
conventional  2-D  ultrasound  imaging  uses  a  hands-on 
interactive  procedure  that  requires  the  operator  to  make 
diagnostic  decisions  simply  in  order  to  place  the  image 
acquisition  probe  atthe.correct  location  and  orientation 
to  see  what  needs  to  be  seen. 

.  For  example,  to  allow  a  diagnosis  of  gallstones 
using  conventional  2-D  ultrasound,  the  operator  must 
interactively  manipulate  the  image  acquisition  probe  so 


as  to  locate  the  gall  bladder,  image  the  bile  duct  at  the 
correct  angle  to  measure  its  diameter,  and  finally  locate 
the  stones  within  the  bladder.  A  positioning  error  of  only 
two  or  three  millimeters,  relative  to  the  patient's  internal 
anatomy,  can  make  the  difference  between  diagnostic 
images  and  useless  ones.  This  need  for  precision 
pointing  makes  it  problematic  to  use  conventional  2-D 
ultrasound  in  a  telemedicine  setting  where  the  diagnostic 
expert  does  not  have  direct  control  over  the  probe 
positioning. 

In  contrast,  using  3-D  (three-dimensional) 
ultrasound  allows  high  quality  scans  to  be  taken  by  an 
operator  with  limited  training  and  no  diagnostic  skills. 

This  is  accomplished  by  having  the  system  scan  a  fairly 
large  volume  of  the  subject’s  anatomy  at  ope  time, 
without  interpretation,  so  that  the  operator  can  use  a 
simple  "point-and-shoot”  strategy  for  data  acquisition. 

For  example,  to  scan  for  gallstones  using  3-D 
ultrasound,  the  operator  has  to  know  only  enough 
anatomy  to  scan  a  volume  that  surrounds  the  gall 
bladder.  Measuring  the  bile  duct  and  locating  individual 
stones  is  still  required,  but  this  analysis  and  diagnosis 
can  be  done  by  an  ultrasound  expert  located  somewhere 
else. 

In  the  summer  of  1996,  a  prototype  3-D 
ultrasound  telemedicine  system  was  developed  for  use 
by  the  U.S.  Army  under  field  conditions.  This  system, 
called  the  MUSTPAC-1,  was  tested  using  a  variety  of 
data  communication  links  between  Germany,  Bosnia, 
and  several  sites  in  the  U.S.  The  system  worked  well, 
experiencing  no  major  failures  and  exceeding 
expectations  in  some  areas.  ■ 

This  report  discusses  design  and  implementation 
of  the  MUSTPAC-1  system  and  summarizes  results  of 
the  Bosnia  field  test. 
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THE  MUSTPAC-1  SYSTEM 


OVERVIEW.  The  name  MUSTPAC-1  is  an 
acronym  for  Medical  UltraSound,  Three  dimensional  and 
Portable,  with  Advanced  Communications  -  1st 
gen^eration.  MUSTPAC-1  is  an  ultrasound  medical 
imaging  system  that  can  scan  patients  to  generate  3-D 
volumetric  digital  datasets,  interactively  generate  3-D 
and  2-D  images  for  use  by  diagnosticians,  and  optionally 
transfer  datasets  over  standard  communication  links  to 
facilitate  remote  diagnosis  and  consultation. 

The  MUSTPAC-1  system  is  optimized  for  use  in 
a  telemedicine  framework.  It  provides  the  unique 
capability  that  high  quality  ultrasound  scans  can  be  taken 
by  an  operator  with  no  diagnostic  skiils,  little  training, 
and  no  online  connection  to  an  expert. 

Typically  MUSTPAC-1  is  used  as  follows.  First, 
the  patient  is  scanned  by  placing  an  ultrasound  probe  on 
the  patient  and  mechanically  sweeping  it  across  their 
skin  over  the  area  of  interest  (Figure  1).  During  the 
scan,  the  system  records  ultrasound  data  from  a  sizable 
3D  volume  of  the  patient's  anatomy,  producing  a  3D 
volumetric  dataset  of  ultrasound  reflectivity.  The 
scanning  process  requires  no  interpretation  of  the 
ultrasound  images,  other  than  possibly  to  confirm  that 
the  intended  anatomy  is  covered. 


Figure  1.  Acquiring  a  3-D  abdominal  scan  using  the 
3-D  Paddle  and  actual  ultrasound  probe. 

Scans  in  the  form  of  3D  volumetric  datasets  are 
then  transmitted  over  any  standard  digital  network  to  a 
qualified  diagnostician. 

Finally,  a  diagnostician  interprets  each  3-D  scan 
using  a  Virtual  Ultrasound  Probe  that  simulates  a 
conventional  real-time  hands-on  examination  procedure. 
This  allows  the  diagnostician  to  display  arbitrary  2D 
slices  from  the  3D  dataset  simply  by  moving  the  probe 
as  if  they  were  interactively  examining  the  patient.  The 
Virtual  Ultrasound  Probe  and  corresponding  screen 
displays  are  very  natural  to  diagnosticians,  leading  to 
rapid  acceptance  and  productivity. 


Figure  2.  Interpreting  a  3-D  scan  using  the  Virtual 
Ultrasound  Probe  at  a  diagnostic  workstation. 


Figure  3.  Closeup  of  the  Virtual  Ultrasound  Probe 
and  diagnostic  workstation  display. 


Figure  4.  Actual  diagnostic  display  (human  liver 
with  clotting  in  major  veins). 
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Figure  5.  MUSTPAC-1  system  components  as 
packaged  for  military  field  evaluation. 

SYSTEM  COMPONENTS  AND  PACKAGING.  Major 
components  of  the  MUSTPAC-1  system  are  shown  in 
Figure  5.  These  components  include  (right-to-left): 

•  Backpackable  field  unit  containing  "3-D  Paddle" 
electromechanical  scanner  (at  extreme  right) 

•  Silicon  Graphics  Presenter™  flat  panel  display  (to 
right  of  backpack) 

•  Hitachi  EUB-905™  ultrasound  machine  (in 
backpack,  top  section,  with  cord) 

•  Silicon  Graphics  Indy™  computer  (in  backpack 
bottom  section) 

•  Teleconferencing  camera  (on  backpack,  top  left) 

•  Keyboard  with  integral  touchpad  (in  front  of 
backpack) 

•  High-resolution  color  monitor. 

•  Virtual  ultrasound  probe 

•  TelelnViVo™  visualization  software  (a  product  of 
Fraunhofer  CRCG,  Providence  Rl,  customized  for 
MUSTPAC-1). 

•  Other  custom  software  for  data  acquisition  and 
control. 


MUSTPAC-1  Principles  cf  Operaticn 


Figure  6.  Major  components,  operations,  and  data 
flows  in  MUSTPAC-1. 


PRINCIPLES  OF  OPERATION.  Figure  6 
diagrams  the  major  components,  operations,  and  data 
flows  within  the  MUSTPAC-1  system. 

Scanning.  Volumetric  scanning  is  done  by 
mechanically  moving  the  probe  of  a  conventional  off-the- 
shelf  ultrasound  machine  (Hitachi  EUB-905).  As  the 
probe  moves,  the  ultrasound  machine  generates  a 
sequence  of  video  images  showing  a  2-D  slice  of  the 
subject  under  the  probe.  A  computer  digitizes  the 
sequence  of  2-D  images  and  assembles  them  to  form  a 
3-D  volumetric  dataset. 

Moving  the  ultrasound  probe  is  accomplished  by 
using  an  electromechanical  device  attached  to  it.  In  the 
MUSTPAC-1,  this  device  is  called  the  "3-D  Paddle".  It  is 
a  battery-powered  device  that  moves  the  probe  under 
motor  control,  in  a  straight  line,  at  a  precise  speed  of  1.0 
cm/second.  This  linear  scanning  movement  generates  a 
set  of  parallel  images  that  naturally  assemble  to  form  a 
dense  rectangular  3D  dataset. 

Scans  are  usually  taken  at  1 5  fps  (frames  per 
second),  using  a  3.5  MHz  or  5  MHz  convex-face  sector- 
scan  probe.  With  typical  penetration  depths,  this  results 
in  scanning  a  region  6  cm  wide  at  the  skin  surface  and 
20  cm  wide  at  15  cm  deep,  with  a  voxel  (volume 
element)  size  of  0.7  mm  on  each  axis. 

it  is  important  to  note  that  the  3-D  datasets 
captured  by  this  process  represent  a  sort  of  “snapshot" 
of  the  subject’s  anatomy.  Anatomical  movement  is  not 
captured,  except  in  the  form  of  motion  artifacts  that  the 
diagnostician  must  learn  to  ignore. 

In  addition  to  the  3-D  ultrasound  dataset 
generated  during  the  scan,  the  MUSTPAC-1  also 
captures  a  single-frame  image  from  each  of  two  attached 
video  cameras.  Typically  these  are  used  to  record 
placement  of  the  3-D  Paddle  on  the  subject's  body. 


Visualization  for  Analysis  and  Diagnosis.  The 
MUSTPAC-1  can  generate  several  kinds  of  images: 

*  2-D  slices  at  arbitrary  positions  and 
orientations. 

*  3-D  volumes  using  "maximum  intensity 
projection"  (MIP)  and  similar  techniques  for 
combining  the  values  of  dataset  volume 
elements  (voxels). 

*  3-D  surfaces  defined  by  threshold  values  and 
rendered  using  one  of  several  shading 
techniques. 

Several  images  can  be  shown  simultaneously. 
The  most  common  practice  is  to  combine  a  large  2-D 
slice  image  with  a  small  3-D  volumetric  image  (Figure  4) 
The  2-D  slice  shows  detail  that  is  useful  diagnostically, 
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while  the  3-D  volumetric  image  provides  context  by 
showing  location  of  the  2-D  slice  within  the  volume. 

Virtual  Ultrasound  Probe.  To  provide 
diagnosticians  with  a  familiar,  convenient  user  interface, 
the  MUSTPAC-1  is  equipped  with  a  Virtual  Ultrasound 
Probe  (Figure  3).  This  is  a  hand-held  6-D  input  device 
that  controls  2-D  slicing  of  volumetric  datasets  in  the 
same  way  that  a  real  ultrasound  probe  controls  2-D 
imaging  of  patients.  That  is,  theVirtual  Ultrasound  Probe 
looks,  feels,  and  acts  like  a  real  ultrasound  probe  in  that 
the  on-screen  image  is  constantly  updated  in  real  time 
(typically  5-10  times  per  second)  to  reflect  the  position 
and  orientation  of  the  probe.  (Again,  this  real  time 
updating  does  not  reflect  anatomical  movement,  which  is 
not  captured  in  the  datasets). 

In  the  MUSTPAC-1,  the  Virtual  Ultrasound  Probe 
is  implemented  using  an  Immersion  Probe™  (product  of 
Immersion  Corporation)  with  a  dummy  ultrasound  probe 
added  to  it.  Other  6-D  input  devices,  such  as  magnetic 
free-space  trackers,  could  easily  be  substituted  for  the 
Immersion  Probe™. 

Data  Communication.  The  MUSTPAC-1  system 
provides  two  major  methods  of  data  communication: 
batch  mode  file  transmission  and  TeleInViVo™ 
incremental  transmission. 

In  batch  mode  file  transmission,  3-D  datasets 
and  associated  files  are  transferred  using  the  Internet 
standard  "ftp"  protocol.  These  transfers  are  for  ease  of 
use  over  relatively  high  bandwidth  communication 
channels.  They  are  initiated  by  the  user  from  a  "desktop" 
graphical  interface  using  a  drag-and-drop  procedure. 

Incremental  communication  capabilities  are  also 
built  into  the  TelelnViVo  visualization  program.  These 
capabilities  allow  all  or  part  of  the  full  dataset  to  be 
transferred  quickly  at  reduced  resolution,  while  the  user 
interactively  evaluates  the  images.  Typical  use  is 
envisioned  to  be  transfer  of  the  entire  dataset  with 
resolution  reduced  by  4X  per  axis  (64X  overall),  followed 
by  transmission  of  an  identified  region  of  interest  at  the 
full  resolution  of  the  dataset. 

Image  Quality.  The  MUSTPAC-1  produces 
diagnostic  quality  images. 

At  maximum  resolution,  the  saved  3-D  datasets 
capture  all  detail  present  in  the  video  output  signal  from 
the  Hitachi  EUB-905  (640x480  pixels  at  8-bit  gray  scale). 
However,  maximum  resolution  capture  often  is  not 
Justified  due  to  noise  and  fuzziness  in  the  ultrasound 
imaging  process. 

Better  results  are  usually  obtained  by  a  simple 
process  that  filters  the  digitized  images  and  samples  the 
filter  output  to  construct  the  saved  3D  dataset.  Operator 


controls  are  provided  to  select  the  amount  of  filtering. 
Typical  settings  produce  one  dataset  voxel  as  the 
average  of  a  2x2  block  of  pixels  from  the  digitized  video 
image. 

There  is  no  image  degradation  due  to  storage 
and  transmission  of  the  3D  datasets.  All  transmission 
protocols  incorporate  error  detection  and 
correction/retransmission.  Only  non-lossy  (exact) 
compression  techniques  are  used  in  the  MUSTPAC-1. 

FIELD  EXPERIENCES.  In  August,  1996,  the 
MUSTPAC-1  system  was  deployed  by  the  U.S.  Army  to 
the  212th  Mobile  Army  Surgical  Hospital  in  Tuzia, 
Bosnia.  A  second  MUSTPAC-1  placed  at  the  Army's 
Landstuhl  Regional  Medical  Center  in  Germany  served 
as  the  primary  receiving  station. 

Scanning.  During  the  deployment,  a  total  of  72 
scans  were  performed  on  42  volunteers,  as  follows: 


Scan  Type 

Number 

Right  Upper  Quadrant 

55 

Pelvis/Uterus/Posterior  Cul-de-Sac 

7 

Placenta 

5 

Renal 

3 

Extremity 

1 

Aorta 

1 

Total 

72 

Most  of  these  scans  were  taken  by  operators 
with  no  ultrasound  diagnostic  skills  and  minimal 
MUSTPAC-1  training  (typically  10  minutes).  No  formal 
evaluation  of  the  image  quality  has  yet  been  performed. 
Informal  evaluation  by  a  variety  of  experienced 
ultrasound  users  suggests  that  the  quality  of  the  scans 
was  largely  independent  of  an  operator’s  level  of  training 
and  generally  ranged  from  adequate  to  very  good. 

Communications.  Most  transfers  between 
Bosnia  and  Germany  were  performed  using  ftp  over  a 
T1/E1  (roughly  1  megabit/second)  geosynchronous 
satellite  link  leased  by  the  Army.  Despite  a  round-trip 
packet  delay  of  580-600  ms,  net  transfer  rates  of  roughly 
50  Kbytes/second  were  routinely  achieved.  Thus  typical 
3-D  datasets  of  6-12  Mbytes  required  only  a  few  minutes 
to  transfer  even  without  compression. 

As  exercises,  3-D  datasets  were  also  transferred 
over  two  slower  communication  links.  One  of  these  links 
was  between  Germany  and  Washington  DC  (USA)  at  56 
Kbits/sec  using  the  International  Maritime  Satellite 
System  (INMARSAT)  and  its  associated  telephone 
system  ISDN  link.  The  other  was  between  Bosnia  and 
Germany  using  the  Army’s  Tactical  Packet  Network 


4 


,  (TACNET)  at  9.6  Kbits/sec.  These  transfers  were  also 
successful,  but  would  be  too  slow  for  most  routine 
applications  without  the  use  of  more  aggressive  data 
compression  techniques. 

Diagnostic  Usability.  During  the  Bosnia 
deployment  and  in  the  6  months  since  its  completion,  the 
MUSTPAC-1  system  has  been  operated  by 
approximately  20  experienced  ultrasound  users.  Informal 
evaluation  suggests  that  the  system  is  very  easily 
learned.  One  striking  observation  is  that  no  experienced 
ultrasound  user  to  date  has  required  more  than  5 
minutes  practice  with  the  Virtual  Ultrasound  Probe  to 
begin  making  medical  interpretations  of  3-D  scans  that 
they  had  not  previously  seen.  However,  no  formal 
studies  of  usability  or  effectiveness  have  been 
performed,  and  such  studies  will  be  needed  to  determine 
the  degree  of  diagnostic  accuracy  provided  by  the 
MUSTPAC’s  techniques. 

Adverse  Events.  During  the  field  test  in  Bosnia, 
one  subject  was  physically  pinched  by  the  3-D  Paddle, 
resulting  in  a  small  superficial  bruise  requiring  no  medical 
treatment.  The  operational  procedure  for  the  3-D  Paddle 
vyas  immediately  revised  to  avoid  a  repetition,  and  the 
3-D  Paddle  has  been  redesigned  to  make  pinching  less 
likely  in  future  designs. 

LIMITATIONS.  While  MUSTPAC-1  provides  a 
unique  capability  for  ultrasound  telemedicine,  it  is  a  first- 
generation  system  with  significant  limitations.  These 
limitations  include: 

•  No  Doppler.  Neither  the  Hitachi  EUB-905 
ultrasound  scanner  nor  any  of  the  MUSTPAC-1 
software  supports  any  Doppler  capability  at  this  time. 
This  means  that  MUSTPAC-1  provides  no  direct 
visualization  of  blood  flow. 

•  Inflexible  scanning.  The  motor-driven  3-D  Paddle  is 
simple  to  use  and  robust,  but  provides  only  linear 
translation  parallel  scanning.  Freehand  scanning 
would  be  more  flexible  and  simpler  to  use  in  many 
parts  of  the  body. 

•  No  anatomic  motion.  Conventional  real-time  2-D 
ultrasound  shows  anatomic  motion,  such  as  pulsing 
arteries,  as  a  moving  image  on  screen.  This  is 
diagnostically  useful.  However,  the  3-D  scans 
captured  by  MUSTPAC-1  are  static  snapshots. 
Generally  speaking,  with  MUSTPAC-1  anatomic 
motion  produces  image  artifacts  that  the 
diagnostician  must  ignore. 

Ongoing  development  of  the  MUSTPAC  system 
is  planned  to  relieve  these  limitations. 


SUMMARY 

3-D  ultrasound  data  acquisition  can  potentially 
enable  effective  use  of  ultrasound  imaging  in  a 
telemedicine  setting,  by  allowing  an  operator  with  no 
diagnostic  skills  to  collect  high  quality  scans  that  can  be 
interpreted  by  a  remote  expert.  This  capability  is 
illustrated  by  the  MUSTPAC-1,  a  portable  3-D  ultrasound 
telemedicine  system  recently  developed  for  the  U.S. 
military  and  successfully  field-tested  by  the  U.S.Army  in 
Germany  and  Bosnia  in  August  1996. 
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DLTRASOnNI)  TELEMEDICINE  SYSTEM  WITH  VIRTOAIi  REALITY 

5  This  invention  was  made  with  Government  support  under 

a  contract  awarded  by  the  U..S.  Department  of  Defense.  The 
Government  has  certain  rights  in  the  invention. 

FIELD-  OF  THE  INVENTION 

The  present  invention  relates  generally  to  a  method 
10  and  apparatus  for  obtaining  medical  ultrasound  images 

remotely  and  manipulating  the  medical  ultrasound  images  for 
diagnosis . 

BACKGROUND  OF  THE  INVENTION 


Ultrasound  imaging  began  in  the  1950 's  and  slowly 
15  developed  from  A-mode  systems  to  real-time  imaging  used 
today  in  obstetrics,  and  diagnosis  and  management  of 
disease.  Presently  in  use  are  ultrasonic  systems  viewing  in 
two  dimensions.  The  two-dimensional  systems  are  dependent 
•  upon  the  experience  and  skill-  of  the  operator  to  obtain  an 
20  optimal  position  of  the  ultrasound  transducer  to  obtain  an 
adequate  image  of  the  anatomy  of  interest.  This  is  because 
of  the  inherent  geometric  constraints  when  using  a  spatially 
flexible  2-D  imaging  technique  to  view  3-D  anatomy.  In 
p^^d^Ice,  the  operator  of  an  ultrasound  transducer  or 
25  scanner  is  .trained  as  a  physician,  radiologist,  or 

ultrasound  technician  with  several  years  of  specialized 
training.  Because  of  the  scarcity  of  skilled  operators, 
patients  are  often  transported,  thereby  delaying 
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examination.  Thus,  whether  the  ultrasonic  probe  is 
proximate  or  remote  to  the  physician,  the  skilled  operator 
must  accompany  the  patient  for  adequate  2-D  images. 

Efforts  have  been  made  to  obtain  ultrasound 
5  examinations  remotely  via  telemedicine.  A  conventional 

real-time  2-D  ultrasound  imaging  system  is  combined  with  a 
video  teleconference 'systeiiri  wherein  an  operator  located  with 
the  patient  positions  the  scanner  and  the  subsequent 
ultrasound  image  is  transmitted  to  the  physician  who  may 
10- -request  changes  in  scanner  or  probe  position  for  optimal 

viewing  of  the  anatomy  of,  interest.  Unless  the  operator  is 
already  trained  to  perform  ultrasound  examinations 
independently,  he  is  dependent  upon  the  physician' s 
instructions  to  position  or  reposition  the  scanner  or  prpbe. 
15  Thus  the  feedback  loop  that  controls  probe  position  based  On 
image  interpretation  includes  two  people  who  must 
communicate  by  language.  Because  the  person  who  inteirprets 
the  images  does  not  have  direct  physical  control  of  the 
probe  position,'  the  time  needed,  to  perform  the  examination. 

20  using. this  approach  is  greatly  increased  and  the  quality  of 
the  exam  may  be  significantly  decreased. 

Research  has  developed  3-D  imaging  techniques  that- 
reduce "variability  of  the  2-D  techniques  and  to  permit  .a 
physician  to  view  3-D  images  via  virtual  ultrasound  probes. 
25  For  data  acquisition,  there  have  been  developed  articulated 
arms,  free-space  magnetic  position  sensors,  circular  arc  ' 
scan,  parallel  scan,  and  image  derived  position.  ■  Others 
have  digitized  ultrasound  data  into  a  3D  digital  database. 
Graphic  visualization  either . as  volumetric  images  or  2D 
30  slices  has  been  achieved.  A  3D  viewing  system  has  been 
developed  for  training  by  MedSim.  In  the  MedSim  viewing 
system,  a  volumetric  ultrasonic  image  is  loaded  onto  a 
computer  and  the  computer  fitted  with- a  virtual  ultrasound 
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probe  implemented  using  a  free -space  magnetic  position 
sensor.  Students  call  up  the  volumetric  ultrasonic  image 
dataset  on  the  computer,  and  with  the  virtual  ultrasound 
probe  controlling  a  2-D  image,  explore  the  volumetric 
5  ultrasonic  image  dataset  on  the  computer  screen.  The  same 
volumetric  image  dataset  is  used  repeatedly  for  numerous 
students .  • 

However,  to  date,  a  system  that  integrates  the 
functions  of  scanning,  transmission,'  reception  and 
10-  -  interpretation  under  a  user  interface  permitting  an 
unskilled  operator  to  quickly  and  easily  obtain  .3D 
ultrasound  datasets  and  transmit  them  to  a  skilled 
diagnostician  for  interpretation/diagnosis  has  not  been 
assembled. 

15  Thus,  there  remains  a  need  for  an  ultrasonic  imaging 

system  that  can  be  operated  by  an -unskilled  operator  yet 
produce  images  interpretable  by  a  skilled  physician. 

SUMMARY  OF  THE  INVENTION 

The  present  invention  is  _an  apparatus  for  ultrasound 
20  telemedicine,  that  combines  3-D  ultrasound  data  acquisition 
with  a  virtual  ultrasound  probe  and  a  communication  link  to 
provide  volumetric  data  that  can  be  viewed  as  either  2-D  or 
3-D  images.  More  specifically,  the  apparatus  has 

(a)  an  ultrasound  probe  having  a  transducer  for 
25  converting  an  electrical  signal  to  an  ultrasonic  signal  and 

for  converting  a  received  reflected  ultrasonic  signal  to  a 
received  electrical  signal; 

(b)  a  position  sensor  providing  spatial 
coordinates  for  a  position  of  the  ultrasound  probe 

30  (c)  a  medical  ultrasound  unit  that  sends  a 

plurality  of  electrical  signals  and  receives  a  plurality  of 
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received  electrical  signals  and  converts  the  plurality  of 
received  electrical  signals  into  a  plurality  of  2-D 
reflected .image  signals; 

(d)  a  first  computer  (on-site  computer)  having.  ■ 
5  software  instructions  that  combines  the  plurality  of  2-D 
reflected  image  signals  into  a  volumized  data  set,  said 
first  computer  further  having  a  communication  link  for 
transmitting  the  volumized  data  set  to  a  remote  location; 
and  •  •  .  ■ 

10-' •  (e)  a  second- computer,  (diagnostic  computer)  at 

the  remote  location  that  receives  the  volumized  data  set 
from  the  communication  link  and  creates  a  three  dimensional 
model  from  the  volumized  data  set. 

The  apparatus  preferably  further  has  a  virtual 
15  ultrasound  probe  connected  to  the  computer  at  the  remote 
location,  for  viewing  cross  sections  of  the  three 
dimensional  model.  An  advantage  of  using  the  virtual 
ultrasound  probe  is  that  the  diagnosing  physician  is  already 
-familiar  with  actual  ultrasound  probes  and  requires  but  a  . 
20  few  minutes  practice  to  effectively  use  the  virtual 
ultrasound  probe. 

'  It  is  an  object  of  the  present  invention  to  provide  an 

apparatus  for  ultrasound  telemedicine  that  permits 
acquisition  of  ultrasonic  image  data  from  a  patient  by  an 
25  unskilled  operator  while  producing  ultrasonic  images 
iriterpretable  by  a  trained  physician. 

The  subject  matter  of  the  present  invention  is 
particularly  pointed  out  and  distinctly  claimed  in  the 
concluding  portion  of  this  specification.  However,  both  the 
30  organization  and  method  of  operation,  together  with  further ■ 
advantages  and  objects  thereof,  may  best  be  understood  by 
reference  to  the  following  description  taken  in  connection 


A2-5 


with  accompanying  drawings  wherein  like  reference  characters 
refer  to  like  elements. 

BRIEF  DESCRIPTION  OF  THE  DRAWINGS 

FIG.  1  is  a  schematic  of  an  ultrasound . telemedicine 
5  system  according  to  the  present  invention. 

FIG',  la  is  an  expanded  schematic  of  the  field 

equipment .  .  .  ' 

.  FIG.  lb  is  an  expanded  schematic  of  the  diagnostic 

equipment. 

10  .FIG.  Ic  is  a.  photo  of  a  computer  screen  capture 

showing  four  views  of  a  3-D  ultrasound  data  set  of  a  phantom 
breast,  with  an  overlapping  view  of  an  orthogonal  view  of  an 
oblique  cut  through  the  3-D  ultrasound  data  set. 

FIG.  2a  is  a  block,  diagram  of  software  and  data  files. 

15  FIG.  2b  is  a  block  diagram  of  software. 

DESCRIPTION  OF  THE  PREFERRED  EMBODIMENT (S) 

Referring"  to  FIG.  1  the  apparatus  of  the  present 
invention  for  ultrasound  telemedicine,  has: 

(a) '  an  ultrasound  probe  100  with  a  transducer 

20  for  converting  an  electrical  signal  to  an  ultrasonic,  signal 

and  for  converting  a  received  reflected  ultrasonic  signal  to 
a  received  electrical  signal; 

(b)  a  position  sensor  (not  shown)  providing 
spatial  coordinates  for  a  position  of  the  ultrasound  probe; 

25  ,  (c)  a  medical  ultrasound  unit  102  that  sends  a 

plurality  of  electrical  signals  and  receives  a  plurality  of 
received  electrical  signals  and  converts  the  plurality  of 
received  electrical  signals  into  a  plurality  of  2-D 
reflected  image  signals; 
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(d)  a  first  computer  103  having  software 
instructions  that  combines  the  plurality  of  2-D  reflected 
image  signals  into  a  volumized  data  set,  the  first  computer 
103  further  having  a  communication  link  104  for  transmitting 

5  the  volumized  data  set  to  a  remote  location;  and 

(e)  a  second  computer  106  at  the  remote  location 
that  receives  the  volumized  data  set  from  the  communication 
link  104  and  creates  a  three  dimensional  model  from  the 
volumized  data  set. 

10.,  .  The  ultrasound  probe  100  may  be  any  ultrasound  probe 

•including  but  not  limited  to  Hitachi  EUP-C324T. 

The  position  sensor  may  be  any  position  sensor' 

•  including  but  not  limited  to  .articulated  arms  (e.g. 

Immersion  Probe,  Immersion  Corporation,  San  Jose,  •  CA)  ,  free- 
15  space  magnetic  position  sensors  (e.g.  Flock  of  Birds  sensor, 
.Ascension  Technology  Corp.,  Burlington,  VT) ,  circular  arc 
scan,  parallel  scan,  and  image  derived  position.  In  a 
preferred  embodiment ,.  the  position  sensor  is  a  parallel  scan 
linear  scanner  that  is  electrically  motor  driven. 

20  Alternatively,  a  2-D  array  of  ultrasound  transducers  with 
physical  or  computational  focusing  (ultrasound  holography) 
may  be  used.,  . 

The  medical  ultrasound  unit  102  may  be  any  medical 
ultrasound  scanner  including  but  not  limited  to  Hitachi  EUB- 
25  905.  ■  • 

The  first  computer  103  may  be  any  computer,  but  is' 
preferably  a  150  MHZ  or  faster,  80  MB  memory  or  larger,  fast 
floating  point  computation,  multiprocessing  operating  system 
with  a  window-based  graphical  user  interface.  Examples 
30  include  but  are  not  limited  to  Silicon  Graphics  Indy 

computer  running  the  Irix  operating  system  and  a  Pentium 
processor  laptop  computer  running  Solaris  or  Windows/NT. 

FIG.  la  shows  additional  details  of  the  first  computer  103 
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and  its  relationship  to  the  medical  ultrasound  unit  102  and 
the  ultrasound  probe  100,  and  the  position  sensor  107, 
(optional .position  sensor  107a  attached  to  the  second 
computer  106)  .  It  will  be  apparent  to  those  of  skill  in  the 
5  art  that  the  medical  ultrasound  unit  102  and  the  first 

computer  103  may  be  either  in  .an  integral  package/housing  or 
in  separate  packages/  housings . 

The  communication  link  104  may  be  any  communication 
link  including  but  not  limited  to  telephone  communication, 
10- ■ satellite  communication,,  radio  communication,  and  a 

combination  thereof.-  A  preferred  communication  link  is 
within  the  first  computer  103,  specifically  a  lOBaseT 
Ethernet . 

The  second  computer  106  may  be  any  computer,  but  is 
15'  preferably  a  computer  as  specified  above  for  the  first 
computer  103.  The  second  computer  106  (FIG.  lb)  has 
communication  .link  software,  preferably  lOBaseT  Ethernet, 
permitting  receipt  of  data  transmitted  from  the  first 

computer  103 .  .  . 

20  The  three  dimensional  model  may  be  a  volumetric  model 

or  a  2D  slice  model  wherein  the  volumetric  model  is  . 

preferred.  .  • 

The  second  computer  106  may  have  attached  a  virtual 
ultrasound  probe  108  permitting  manipulation  of  a  2-D  plane 
25  through  the  three  dimensional  (3-D)  model  on  the  display  110 
of  the  computer.  The  virtual  ultrasound  probe  108  is  a 
multi-degree,  preferably  6-degree,  of  freedom  spatial 
positioning  input  device  and  associated  software,  for 
example  an  articulated  arm,  free -space  magnetic  position 
30  sensor,  or  other  hardware  familiar  to  ultrasound 

diagnosticians.  Specifically,  an  articulated  arm  was 
obtained  and  fitted  as  the  virtual  ultrasound  probe  108. 
Specifically,  an  Immersion  Probe  with  a  stylus  at  the 
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moveable  end  of  the  articulated  airm  was  modified  by 
■  replacing  the  stylus  with  a  mock-up  of  an  ultrasound  probe. 
In  addition,  the  base  that  the  arm  is  attached  to  was 

I 

replaced  to  maintain  position  calibration.  Finally  an 
5  interface  software  program  was  added  to  transfer  x, ,  y,  z, . 
pitch,  'roll,  and  yaw  from  the  mock-up  ultrasound  probe  to 
the  cutting  plane  on  the  computer  screen. 

FIG.  Ic,  upper  left,  shows  volumetric  visualization 
111  of  an  entire  3-D  dataset  112  with  a  2-D  cutting  plane 
1.0_.ll4.  The  location  of  the  2-D  cutting  plane  114  within  the 
3-D  dataset  112  is  indicated  by  outline  and  also  by 
emphasizing  the  cross  section  defined  by  the  cutting  plane. 
Through  software  interpretation,  the  logical  position  and 
orientation  of  the  cutting  plane  is  made  to  track  the 
15  physical  position  and  orientation  of  the  virtual  ultrasound 
probe  input  device,  so  that  as  the  .user  moves  the  probe 
left,  the  cutting  plane,  moves  left,  and'  so  on.  The  image  is 
of-  a  breast  phantom  of  silicone  plastic  with  simulated  cysts 
molded  therein'.  .  Additionally,  FIG.  Ic,  upper  right  120, 

20  lower  left  122  and  lower  right  124  are  views  orthogonal  to  , 
the  y-z,  x-z,  and  x-y  planes  respectively.  In  the  extreme 
upper  right  of  FIG.  Ic  is  a  line  drawing  125  showing  the  . 
view  angle  of  the  2-D  cutting  plane.  Also 'shown  overlapping 
(fig.  Id)  is  a  perpendicular  or  orthogonal  view  12  6  of  the 
25  cross  section  116;  the  perpendicular  view  126  avoi'ds  image 

distortion  compared  to  an  angular  view.-  The  orthogonal  view 
is  generated  by  a  software '  instruction  set,  preferably  part 
of  the  visualization  software  (FIG.  2a  208,  206)  .  In  FIG. 
Ic,.  the  upper  left  window  shows  a  volumetric  rendering  of 
30  the  entire  3D  dataset,  with  the.  position  of  the  cutting 
plane  The  largest  window  shows  the  cross  section  in 
perpendicular  view,  so  that  it  is  not  distorted. 
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Both  computers  103,  106  have  software  instruction  sets 

•  (FIG.  2a)  for  capturing,  displaying,  transferring/sharing, 
and  printing  graphical  data.  The  medical  ultrasound  unit 

I 

102  generates  2-D  data  in  the  form  of  a  slice  through  an 
5  anatomical  feature.  By  collecting  a  plurality  of  2-D  data 
sets  in  the  form  of  substantially  contiguous .  slices ,  a  3-D 
data  set  can  be  obtained.  Each  2-D  slice  or  frame  may  have 
640  X  480  pixels  with  30  frames  per  second. 

The  medical  ultrasound  unit  102  converts  ultrasonic 
IQ,  .  signals  into  digital  data  in  a  proprietary  format  then 

converts  the  proprietary  format  digital  data  into  a  standard 
analog  video  format.  Data  from  the  medical  ultrasound  unit 
102  -may  be  transferred  to  the  first  computer-  103  either  as 
proprietary  format  digital  data  or  in  standard  analog  video 
15  format.  While  it  is  simpler  to  use  the  standard  analog 

video  format,  greater  quality  of  imaging  is  achieved  using 
the  proprietary  format  digital  data  directly J 

When  the  standard  a.nalog  video  format  is  used,  the 
frames  are  captured  on  a., video  capture  card  .200  within  the 
20  first  computer- 103 .  The  video  capture  card  may  be  any 

standard  video  capture  card,  for  example  a  Silicon  Graphics 

■  VINO  subsystem.  Optionally,  the  second  computer  106  may. 
have  a  video  capture  card  200a.  Standard  video  capture 
software  is  able  to  receive  frame  data  from  the  video 
25  capture  card  and  display  the  2-D  image.  However,  that  is  not 
sufficient  for  3-D  visualization  or  rendering.  Accordingly, 
a  standard  video  capture  software  was  modified  to  include 

•  storing  each  2-D  data  frame  and  volumizing  the  2-D  data 
frames  to  obtain  a  'l-D  "volumized  data  set,  as  a  video 

30  capture  and  volumization  software  202,  202a.  The  3-D 

volumized  data  set  is  preferred  because  it  is  a  standardized 
geometry  including  scaling  parameters.  The  video  capture 
software  was  prepared  from  Silicon  Graphics'  online 
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reference  documentation,  IRIS  Digital  Media  Programming 
Guide,  Chapters  11-14  on  an  Indy  computer  having  a  VINO 
(Video  Input  No  Output)  subsystem. 

Whether  from  proprietary  format  digital  data  or  from 
5  standard  analog  video  format,  data  volumization  from  2-D 
slices  is  preferably  done  according  to  the  following  steps: 

(a)  establish  scaling  parameters  that  describe  voxel 
sizes,  in  other  words,  number  of  pixels  per  unit  length 
within  each  frame  and  separation'  unit  length  between  frames; 
10-  •  (b)  filter  the  image  to  reduce  the  volume  of  data; 

and  .  ■  •  ■  . 

(c)  write  the  filtered  data  (3-D  volumized  data  set) 
(e.g.  VOLl  file  format)  to  disk  with  headers  and  trailers' 

'  describing  image  size  and  scaling  in  a  format  compatible 
15  with  the  visualization  software ' 206 . 

Pixels  per  unit  length  may  be  determined  by 
examination  of  scale  lines  and  tick  marks  inserted  into  the 
image  data  by  the  medical  ultrasound  unit.  The  examination 
may  be  manual  supported  by  a  graphical  user  interface,  or 
2.0  automatically,  by  software' providing  image  interpretation. 
Separation  unit  length  may  be  determined  by  the  speed  of  a 
position  sensor  and  the  frame  capture  rate. 

Filtering  of  the  data  may  be  done  by  computing  spatial 
averages  within  a  neighborhood  of  pixels  within  each  2-D 
25  image  and/or  between  nearby  frames,  storing  only  the 

averages  instead  of  all  captured  pixel  values .  The  exact  . 
settings  (filter  parameters)  are  preferably  operator 
controlled/  selected. .  Preferred  settings  are  averaging  2X2 
blocks  of  pixels  within  each  frame,  and  collecting.  15  frames 
30  per  second  with  no  averaging  resulting  in  an  8-fold 

reduction  of  data  compared  to  using  all  pixels  captured  at 
30  frames  per  second.  Further  data  compression  may  be 
accomplished  using  more  sophisticated  methods,  including  but 
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not  limited  to  wavelet  compression -techniques ,  for  example 
Joint  Photographies  Experts  Group  (JPEG)  compression. 

Once  the  3-D  volumized  data  set  is  made,  there  are  two 
•  options  with  respect  to  further  processing  to  produce 
5  viewable  images:  (1) ■ transmit  the  3-D  volumized  data  set  via 
file  transfer  software  204  in  the  first  computer  103  to  the 
file  transfer  software  205  in  the  second  computer  105,  (2) 
use  visualization  software  208  in  the  first  computer  103  to 
create  a  3-D  visualization  data  set  in  the  first  computer 
IG' -103,  then  connect  with  the  visualization  software  206  in  the 
second  computer  106  and' share- the  3-D  visualization  data  set 
between  both  computers .  A  further  option  is  encryption  of 
either  (1)  or  (2)  prior  to  transmission  or  sharing  with  - 
enczyptidn  software  210,  212.  Visualization  software  must 
15  be  visualization  software  which  has  an  interface  with  a 
virtual  ultrasound  probe  108,  'for  example  TeleInViVo, 
Fraunhofer  CRCG,  Providence,  RI .  Whichever  visualization 
software  is  used,  the  3-D  volumized  data  set  must  be  in  a- 
format  compatible  with  the  visualization  software .  Once  a 
20  3-D  visualization  data  set  has  been  created,  the 

visualization  software  206,  208  may  call  the  3-D 
visualization  data  set  as  many  times  as  desired  for 
observation  of  the  graphical  image  ori  a  screen.  The  screen 
view  is  updated  in  real  time  (5-10  updates  per  second)  as  ■ 

25  the  virtual  ultrasound  probe  108  -is  moved. 

Encryption  software  210,  212’  may  be  any  encryption 
software,  for  example  as  available  from  RSA  Redwood  City,  CA 
or  PGP  San  Mateo,  CA.  In  a  preferred  embodiment,  the 
encryption  software  210,  212  is  capable  of  international  use 
30  inasmuch  as  the  location  of  an  injured’ or  .disabled 

individual  may  be  in  a  different  country  compared  to  the 
remote  location  of  a  diagnosing  physician. 
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Beyond  sharing  image  data,  it  may  be  necessary  for  a 
diagnosing  physician  at  the  .remote  location  to  communicate 
observations,  instructions  or  interpretations  to  the  on-site 
operator.  Accordingly,  both  computers  103,  106  preferably 
include  videoconferencing  software  230,  232. 

Specific  instructions  for  each  of  the  four  functions 
of  capturing,  displaying,  transferring/sharing,  and  printing 
are  preferably  encoded  in  a  general  graphical  user  interface 
(GUI)  ■  FIG.  2b  214,  216.  Options  include  an  on-line  tutorial 
10- -218,  220,  printer  software  222,  224,  and  a  medical  library 
226'.  The  printer  software  222,  224  preferably  is  an 
interface  permitting  a  user  to  print  or  output  a  screen, 
window,  body  of  text,  and/or  graphic  or  picture.  The  print 
may  be  directed  to  a  file' or  a  printer. 

15  The  graphical  user  interface  214,  216  is  a  captured 

user  environment,  controlled  by  a  user  login  ID,  and  driven 
by  the  operating  system  native  desktop  manager.  The 
graphical  user  interface  214 ,  216  has  icons  for  operating 
the  videoconferencing  software  23  0,  232,  video  capture  ; 

20  software  202,  visualization  software  206,  208,  file  transfer 
software  204,  20.5,  encryption  software  210,  212,  on-line 
tutorial  218,  220,  printer  software  222,  224,  and  medical 
library  226,  228.  The  graphical  user  interface  214,  216 
further  includes  a  file  manager  (not' .shown)  permitting 
25  navigating  though  directories,  manipulating  (finding, 

creating,  copying,  moving,  deleting)  files,  dragging-  and 
dropping  between  and  within  windows .  The  graphical  user 
interface  214,  216  further  includes  archiving  for  storing 
ultrasound  datasets  in  random  access  removable  media,  as 
30  well  as  documenting  for  storing  textual  information  with  the 
ultrasound  data  sets,  and  video  monitor  controlling  for 
switching  monitor  configurations  for  alternate  displays. 
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Although  the  invention  has  been  described  .in  terms  of 
3  spatial  dimensional  data,  it  will  be  apparent  to  one  of 
skill  in  the  art  that  time  dependent  data  may  also  be  used. 

t 

The  hardware  would  be  the  same,,  and  the  software  would 
5  simply  be  modified. to  accept  the  time  parameter. 

Closure 

While  a  preferred  embodiment  of  the  present  invention 
has  been  shown. and  described,  it  will  be  apparent  to  .those 
.  .skilled  in  the  art  that  many  changes  and  modifications  may 
10  be  made  without  departing  from  the  invention  in  its  broader 
aspects.  The  appended  claims  are  therefore  intended  to 
cover  all  such  changes  and  modifications'  as  fall  within  the 
true  spirit  and  scope  of  the-  invention. 
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CLAIMS 


We  claim: 

1.  An  apparatus  for  ultrasound  telemedicine, 
comprising: 

5  (a)  an  ultrasound  probe  having  a  transducer  for 

converting  an  electrical  signal  to  an  ultrasonic  signal  and. 
for  converting  a  received  reflected  ultrasonic  signal  to  a 
received  electrical  signal; 

(b)  a  position  sensor  providing  spatial 

10' • coordinates  for  a  position  of  the  ultrasound  probe; 

(c)  a  medical  ultrasound  unit  that  sends  a 
plurality. of  electrical  signals  and  receives  a  plurality  of 
received  electrical  signals  and  converts  the  plurality  of 
received  electrical  signals  into  a  plurality  of  2-D 

15  reflected  image  signals; 

(d)  a  first  computer  having  software 
instructions  that  combines  the  plurality  of  2-D  reflected 
image  signals  into  a  volumized-  data  set,  said  first  computer 
further  having  a  communication  link  for  transmitting  the 

20  volumized  data  set  to  a  remote  location;,  and 

(e)  a  second  computer  at  the  remote  location 

that  receives  the  volumized  data  set  from- the  communication 
link  and  creates  a  three  dimensional  model 'from  the 
volumized  data  set..  ■  ■ 

25  2.  The  apparatus  as  recited  in  claim  1,  further 

comprising  a  virtual  ultrasound  probe  connected  to  the 
computer  at  the  remote  location,  for  viewing  cross  sections 
of  the  three  .dimensional  model. 

30  '3.  The  apparatus  as  recited  in  claim  1,  wherein  the 

software  instructions  comprise  a  video  capture  and 
volumization  software  that  receives  the  plurality  of  2-D 
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reflected  image  signals,  as  a  plurality  of  frames,  each  of 
the  plurality  of  frames  volumized  with  respect  to  adjacent 
frames  to  produce  the  volumized  data  set.  .  • 

4.  The  apparatus  as  recited  in. claim  3,  wherein  the 
■5  software  instructions  further  comprise  a  3-D  visualization 

software  that  receives  the  volumized  data  set  and  produces  a 
3-D  visualization  data  set.  . 

5.  An  apparatus  .for  viewing  a  2-D  cross  section  of  a 
3-D  dataset  comprising: 

10  (a)  a  diagnostic  computer  with  a  screen  for 

displaying  the  3-D  data  set,  the  diagnostic  computer  further 
having  software  for  generating  a  2-D  plane  viewable  on  the 
screen  simultaneously  with  the  3-D  dataset;  , 

(b)  ■ a  virtual  ultrasound  probe  with  an  interface 
15  software  program  that  transfers  physical  position  of  the 

virtual  ultrasound  probe  to  the  logical  position  of  the  2-D 
plane  defining  a  2-D  cross  section  through  the  3-D  dataset; 

(c)  a  software  instruction  set  permitting  an 
orthogonal  view  of  the  2-D  cross  section  of  the  3-D  dataset 

20  at  the  location,  of  the  2-D  plane.  . 

6'.  '  The  apparatus  as  recited  in  claim  5,  further 
comprising  an  apparatus  for  ultrasound  telemedicine, 
comprising: 

(a)  an  ultrasound  probe  having  a  transducer  for 
25  converting  an  electrical  signal  to  an  ultrasonic  signal  and 

for  converting  a  received  reflected  ultrasonic  signal  to  a 
received  electrical  signal; 

(b)  a  position  sensor  providing  spatial 
coordinates  for  a  position  of  the  ultrasound  probe; 
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(c)  a  medical  ultrasound  unit  that  sends  a 
plurality  of  electrical  signals  and  receives  a  plurality  of 
received  electrical  signals  and  converts  the  plura.lity  of 

I 

received  electrical  Signals  into  a  plurality  of  2-D 
5  reflected  image  signals;  and 

(d)  an  on-site  computer  having  software 
instructions  that  combines  the  plurality  of  2-D  reflected 

signals  into  a'  volumized.  data  set/  said  first  computer 
further  having  a  communication  link  for  transmitting  the 
IQ,  .volumized  data  set  to  the  diagnostic' computer . 

7 .  The  apparatus  as  recited  in  claim  6 ,  wherein  the 
software  instructions  comprise  a  video  capture  and 
volumization  software  that  receives  the  plurality  of  2-D 
15  reflected- image  signals  as  a  plurality  of  frames,  each  of 
the  plurality  of  frames  volumized  with  respect  to  adjacent 
frames  to  produce  the  volumized  data  set . 

•  8 .  The  apparatus  as  recited  in  claim  7 ,  wherein  the 
software  instructions  further  comprise  a  3-D  visualization 
20  software  that  receives  the  volumized  data  set  and  produces  a 
3-D  visualization  data  set.  . 
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Original  portrait  photograph  by  Jan  Jackson;  original  product  photographs  by  Walter  Calah: 


IMAGINE  A  DOCTOR  IN  NEW  YORK 

examining  a  wounded  soldier  in  Bosnia 
with  ultrasound.  All  the  technology 
needed  to  perform  such  a  feat— portable 
ultrasound  devices  and  the  means  for 
transmitting  the  data  from  them — are 
available  right  now.  There’s  just  one 
problem,  says  Rik  Litdefield — ^most  doc¬ 
tors  wouldn’t  know  how  to  use  the  equip¬ 
ment.  lb  sift  through  transmitted  images 
displayed  on  a  monitor,  the  doctor  in 


Littlefield  had  a  prototype  tested  last 
August,  and  now  he  is  readying  a  final 
unit  to  submit  to  the  U.S.  Food  and  Drug 
Administration  for  approval.  Although 
the  military  will  probably  make  the  first 
units  for  the  battlefield,  rural  health  care 
is  likely  to  benefit  as  costs  come  down  in 
a  few  years.  “I’ve  already  started  getting 
calls  from  doctors  in  Alaskan  bush  com¬ 
munities  and  from  Native  American 
reservations,”  says  Littlefield. 


information — about  as  much  as  you’d 
find  on  the  typical  business  card.  Last 
year,  now  as  a  researcher  at  IBM’s  Al- 
maden  Research  Center  in  San  Jose,  Cal¬ 
ifornia,  he  added  some  security  features 
and  made  it  small  enough  to  fit  in  a 
pocket.  “I  want  to  replace  all  the  cards 
you  have  in  your  wallet  with  this  one  de¬ 
vice,”  he  says. 

Unlike  the  free-flowing  electrons  of 
an  electric  current,  the  static  electric  field 


o 


New  York  would  have  to  master  arcane 
computer  keyboard  and  mouse  com¬ 
mand.  “It’s  nonintuitive  and  ineffective,” 
he  says.  “Most  doctors  just  wouldn’t 
bother  to  learn  how.” 

Littlefield,  a  computer  scientist  at  Pa¬ 
cific  Northwest  National  Laboratory  in 
Richland,  Washington,  has  put  together 
a  portable  ultrasound  system  that  makes 
the  doctor  in  New  York  feel  as  though  he 
is  actually  on  the  battlefield.  The  field 
operator— who  requires  only  a  modest 
amount  of  training — positions  a  probe 
over  the  patient’s  wotmd  and  then  the 
system  mechanically  scans  inside  the  pa¬ 
tient.  The  3-D  data  are  then  transmit¬ 
ted  to  a  computer  back  at  the  hospital. 
There  the  doctor  holds  an  input  device 
that  looks  and  feels  like  the  handheld 
probes  that  doctors  ordinarily  use  on  ul¬ 
trasound  patients.  As  the  doctor  moves 
and  twists  the  ersatz  probe  in  midair, 
motion-trackmg  sensors  relay  position¬ 
ing  data  to  the  computer,  which  responds 
by  changing  the  view  on  the  doctor’s 
screen  as  though  the  patient  were  lying 
there  in  front  of  him.  “Of  the  two  dozen 
doctors  who  have  tried  it,”  says  Little¬ 
field,  “not  one  has  t^en  more  than  five 
minutes  to  get  conifortable  enough  to 
make  a  diagnosis.  The  old  system  took 
me  hours  to  learn,  and  I’m  supposed  to 
be  an  expert.” 
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Appendix  4: 

“Ultrasound  marches  to  the  front”,  Portable  Desig 
Magazine  cover  article,  June  1997 


DEV  E  L  O  P  M  E  N  T 


STRATEGI  ES 


Ultrasound  marches 
to  the  front 

.  John  K  MayeVj  Contributing  Editor 


A  multidisciplinary 
design  team  uses  off-the- 
shelf  subsystems  to 
rapidly  develop  a  portable 
telemedicine  box. 


In  military  medical  care,  it’s  Somatic  that 
what  happens  duiing  the  golden  hour — 
the  60  minutes  immediately  following  a 
serious  wound  or  injury — ^is  key  to  patient 
survival.  Unfortunately,  sophisticated  di¬ 
agnostic  equipment,  crucial  to  treat  life- 
threatening  wounds  swiftly,  has  been  too 
cumbersome  to  take  into  the  field,  and 
the  golden  hour  often  passes — ^along  with  the  casu¬ 
alty.  What’s  needed  is  better  portable  medical  gear. 

Now,  a  joint  effort  by  researchers  at  Pacific 
Northwest  National  Laboratory  (Richland,  Wash.) 

changes  the  portable  picture. 
Working  with  medical  tech¬ 
nology  experts  at  the  De¬ 
fense  Advanced 
.Research  Pro¬ 
jects  Agency 
(DARPA)  and 
designers  at  the 
U.S.  Army 
Medical  Re¬ 
search  and  Materiel  Command,  a  de¬ 
sign  team  has  developed  just  such  a  sys¬ 
tem — a  portable  ultrasound  imager.  It’s 
purpose  is  to  bring  the  benefits  of  so¬ 
phisticated  ultrasound  imaging— con¬ 
ventionally  used  in  hospitals — to  the 
front  lines  in  combat  situations. 

Commercial  potential 

Promismg  to  someday  reduce  the  num¬ 
ber  of  battlefield  deaths,  while  offering 
tremendous  commercial  potential,  the 
joint  design  effort  resulted  in  the  design 
of  what’s  called  a  Medical  Ultrasound, 
Three-dimensional  and  Portable  with 
Advanced  Communications  box  (any 
GI  will  tell  you  it’s  easier  to  call  it  by  its 
acronym:  MUSTPAC- 1 ) . 

"‘If  a  patient’s  bleeding  in  the  field 
or,  for  that  matter,  in  some  remote 
county  in  West  Virginia,  I  don’t  have 
time  to  take  him  to  an  MRI,”  says 
Dr.  Christian*  Macedonia,  principal 
medical  investigator  on  the  project. 


“The  direction  medical  diagnostics  has  to  go  is 
toward  making  systems  that  are  lightweight  and 
portable  in  design  so  you  can  take  them  to  the  pa¬ 
tient.”  Macedonia  is  also  a  major  with  the  U.S. 
Army  Medical  Corps  at  Georgetown  University 
Medical  Center,  so  he  knows  of  what  he  speaks. 

Ease  of  use  is  key 

.  The  MUSTPAC-1  permits  a  semiskilled  operator  to 
perform  3-D  scans  of  an  injured  soldier  ^d  send 
those  images  for  interpretation  to  experts  anywhere 
in  the  world.  Macedonia  is  credited  with  much  of 
the  initial  vision  for  MUSTPAC.  In  fact,  he  began 
exploring  the  use  of  3-D  ultrasound  while  he  was  a 
medical  student  in  the  early  19905. 
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STRATEGIE  S 

At  that  time,  Macedonia  was  interested  in  ex-  ultrasound  images,”  explains  Rik  Littlefield,  project 
ploring  how  different  chest  ventilation  methods  af-  manager  at  Pacific  Northwest  National  Laboratory, 
fected  blood  flow  throughout  the  body.  While  he  Littlefield  notes  that  a  classic  ultrasound  exam  is 
uncovered  significant  research  on  Doppler  ultra-  performed  by  a  technician  viewing  a  real-time,  15- 
sound,  one  of  his  criticisms  was  that  it  couldn’t  be  firame/sec  image  as  he  moves  a  probe  around  the 
used  to  track  moving  objects.  ‘‘It  occurred  to  me  body  Without  that  ability  and  without  the  knowl- 
that  we  have  radar  systems  that  can  track  missiles  in  edge  of  how  to  interpret  those  images  and  where  to 
flight,  so  why  can’t  we  apply  some  defense  technol-  re-point  his  hand  to  see  what  he  really  needs  to  see, 
ogy  toward  this  problem.^”  the  operator  can’t  perform  the  exam. 

It  took  a  few  years,  but  Macedonia  eventually  “^^at  3-D  data  collection  permits  is  to  collect 
hacked  together  a  prototype  that  embodied  his  perfectly  useful  data  in  a  sort  of  blind  snapshot 
ideas.  Using  off-the-shelf  components,  he  pieced  to-  mode,”  says  Littlefield.  “If  you  think  someone  nfight 
gether  a  Macintosh  7100  computer,  a  trackball  key-  have  a  gallstone,  obviously  the  person  who  collects 
board,  and  an  ultrasound  machine.  He  used  a  scan  the  data  has  to  know  where  the  gall  bladder  is.  But 
converter  to  transform  the  Macintosh  video  output  he  doesn’t  have  to  know  what  angle  to  point  the 
to  NTSC  video  and  then  ran  it'back  out  the  moni-  probe  in  order  to  image  the  duct  at  the  right  angle 
tor  of  the  ultrasound  machine.  Did  it  work?  “Yes,”  to  take  the  measurements.” 
tells  Macedonia,  “but  it  generated  images  that  were  In  the  MUSTPAC,  a  specially-designed  simu- 
nice  to  look  at,  but  weren’t  particularly  useful.”  lated  probe  looks,  feels,  and  acts  like  a  real  ultra- 

Knowing  he’d  reached  his  technical  limits,  Mace-  soimd  probe.  Littlefield  adds  that  once  an  image  is 

donia  took  his  idea  to  developers  at  Pacific  North-  stored  in  a  3-D  dataset,  it  can  later  be  reviewed  either 
west  National  Laboratory,  operated  by  Batteile  for  locally  or  remotely, 
the  U.S.  Department  of  Energy  (DOE).  Shortly 

thereafter,  the  DOE  and  the  Department  of  De-  Fourth  of  July  prototyping 

fense  (through  DAEPA)  agreed  to  put  up  $5  million  .  Once  funding  was  approved  for  the  project  (that 
to  develop  a  3-D  ultrasound  system  for  diagnosing  happened  in  April  of  1996),  development  of  the 
battlefield  injuries.  MUSTPAC  proceeded  at  what  had  to  be  a  rapid 

pace.  Driving  the  team  was  a  schedule  that  called  for 
A  MUSTPAC  must  delivery  of  a  prototype  by  early  July. 

While  conventional  ultrasound  is  a  2-D  imaging  pro-  Much  like  Macedonia’s  original  design,  MUST- 

cedure,  3-D  ultrasound  was  critical  to  the  premise  PAC-1  designers  built  the  unit  around  oflF-the-shelf 
behind  MUSTPAC — ^scanning  a  fairly  large  volume  components.  The  team  also  imposed  a  requirement 
of  the  patient  at  a  remote  site  by  an  unskilled  oper-  that  the;  system  be  transportable  by  a  single  soldier, 
ator  and  then  transferring  the  data  in  a  store-and-for-  “As  an  officer  in  the  First  Infantry  Division  in 
ward  file  transfer  mode  to  a  skilled  diagnostician.  Germany,  I  clearly  remember  that  the  most  useful 
“The  point  of  the  3-D  is  that  it  permits  data  to  be  •  equipment  was  that  which  one  person  could  carry,” 
collected  by  someone  who  knows  a  little  anatomy,  notes  Macedonia.  “If  it’s  small  enough,  it  can  go  prac- 
but  doesn’t  know  anything  about  how  to  interpret  tically  anywhere.” 


Anatomy  of  a  telemedicine  box 


to  acquireS-D  ultrasound  ;(;^^isualize^|!^^ 
it^lo^ily^a^ f r^sm  dat^o  arer^  siteJU^^i^djan 

I 

Si^cbnsultatibri  ih  a  feiecbhferericei^thaM  tSIiKII 


2-D  ultrasound 


Video  capture 


S' is  powered  a  corfirhercial 
mptible  paw 

jS;45  miriiies:  Ittiahalsb  be  powered^ft^* 

: J  .line.  The  core  of  the  system  is;a  Hitachi 
’>  Systems  Model.  EUB-905  2-0 'ultrasound  sub^-’ 

pbiB-QO^Iinks  to  a  Silicon. Gr^hics 
*  ^woikstaijon  by ' means  of  a  75-bhm  video  cable. '  Pf. 

Video-capture  software  feeds  a  3-D  dat^t  into  .  .;'T 
'  a  volume  data  file  residing  on  the  Indy’s  h^  i- 
.  disk.  Rendering  software  then  permits  operators  to  ^itr^ly'  ' ferf^ 

■  re-slice  the  3-D  dataset  at  any  position  or  orientation  using  a  -  -  A  stan 
'  stand-alone,  joysticklike  virtual  ultrasound  probe.  Data  is  trans-  •'  ■  •  * 


Volume  data  file']^ 


Rendering  software: 


Iilfi  Display  3-D  volumetric  and  2-D  slices  ~  i 

H  (arbitrary  resl^)  _  ■ 


Virtual  ultrasound  probe 
(immersion  probe) 


Telecommunication: 
file  transfer  interactive 
^  collaboration  '■  >  ^3- 


feried  to  rernote  sites  via  a  standard  Ethernet  conn^on. 
A  stand-alone  camera  supports  videoconferencing.  - 
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It  was  decided  to  make  the  system  jSt  into  a  back¬ 
pack,  “That  meant  it  clearly  had  to  be  under  100 
pounds,”  recalls  Littiefield.  What  the  team  ultimately 
designed  weighed  in  at  85  pounds. 

Close  to  25  pounds  of  the  total  weight  was  attrib¬ 
utable  to  the  centerpiece  of  the  system,  an  off-the- 
shelf  EUB-905  ultrasound  subsystem  from  Hitachi 
Medical  Systems  (Tarrytown,  N,Y.).  Measuring  13.6  x 
16  X  6.4  inches  in  size  (34  x  40  x  16  cm)  the  compact 
2-D  unit  offered  a  variety  of  display  modes,  as  well  as 
192  channel  probes,  256  gray¬ 
scale  steps,  and  a  high-fre-  - ^ — 

quency  probe  that  operates  at 
up  to  10  MHz.  It  can  also  be  ■  Bi  V  I 

{xjwered  off  either  a  120-V  ac  IE  iF  ■■  I 

line  or  batteries.  Ill  HI 

‘We  picked  the  EUB-905  ^  » , 

because  it  offered  an  excellent  |f  | 

combination  of  image  quality  U  E 


eral  pounds  of  longitudinal  thrust  and  withstand 
pushing  against  a  patient’s  body. 

Weight  versus  Mips 

The  second  heaviest  component  in  the  MUSTPAC- 
1  is  its  compute  platform.  For  that  element,  the  de¬ 
sign  team  chose  an  Indy  workstation  from  Silicon 
Graphics  Inc.  (SGI — ^Mountain  View,  Galif.).  Built 
around  a  64-bit  MIPS  R5000  microprocessor,  the 
SGI  workstation  is  optimized  for  compute-intensive 


(grated 

Whe^bard  will 
t  (-■  Ui^iy  elimifutte 
htitk  cfthe 
Y  siand--(ilone 
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and  size  and  weight,”  explains 
.  litdefield,' Also  important  to 
the  selection  of  the  EUB-905 
was  that  it  had  won  Food  and 
Drug  Administration  FDA 
510(k)  approval. 

With  future  commercial¬ 
ization  on  their  minds, 
MUSTPAC  designers  went 
out  of  their  way  to  avoid  cus¬ 
tomizing  the  subsystem.  ‘We 
avoided  like  the  plague  break¬ 
ing  into  that  system,”  explains 
Littlefield.  “Doing  anything 
at  all  to  it  that  could  affect  its 
performance  would  have 
voided  the  5I0(k)  approval.” 

Complementing  the 
EUB-905  is  a  3-D  paddle 
electromechanical  scanner. 
Compact  and  lightw'eight,  the 
paddle  was  designed  to  be 
strong  enough  to  support  sev- 


IF  YOU'RE  EMBEDDINC  A 
BUNCHA  BOARDS, 
TAKE  A  CLOSE  LOOK 


No,  it's  not  a  very  large  chip,  it's  a  very 
small  computer:  TheZF  OEModuIe.  It's 
a  complete  computer  that  mounts,  on 
your  board  like  a  chip. 

The  OEModuIe  is  a  complete  386SX  computer 
with  a  form  factor  just  2.20  x  3.00  x  .45  inches 
(55.9  X  76.2  X  11.3  mm). 

Forget  unreliable  cables  and  vibration 
problems. 

Everything  you  need  is  built  into  the 
OEModuIe,  It  boots  form  it's  embedded 
BIOS  and  comes  with  DOS..  Two  COM 
ports,  a  parallel  port,  keyboard  and  speaker 
ports,  IDE  and  floppy  disk  support,  DRAM 
controller  supports  up  to  16  Megabytes,  full 
core  logic,  even  an  internal  300K  Resident 
Flash  Disk  It's  all  there.  Mounted  directly 
on  your  proprietary  board,  connections  are 
rock  solid  even  in  harsh  environments. 


A  familiar  development  environment. 
The  OEModuIe  is  fully  PC/AT  compatible. 
Your  development  tools  will  work  perfectly. 
A  complete  development  kit. 

The  ZF  Microsystems  SBX7386  develop¬ 
ment  board  includes  every  thingyou  need: 
hardware,  software,  and  manuals.  The 
SBX/386  is  compatible  with  both  AT  and 
PC/104  expansion  busses.  That  means  you 
go  from  concept  to  market  in  record  time. . 

Questions?  Call  us,  or  visit  our  site: 
f'..  ■  www.zfmlcro.com 


ZF  Microsystems,  Inc. 
1052  Elweil  Court 
Palo  Alto,  CA  94303 
Tel:  415-965-3800 
Fax:  415-965-4050 
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EVE  LOPMENT  STRATEGIES 


^^3-D  permits  data  to  be  collected  by  someone  who  knows  a 
little  anatomy j  bnt  doesn^t  know  anything  about  how  to 
interpret  tdtrasound  ima^es/^  explains  Rik  Littlefieldy 
project  manager  at  Racijic  Northwest  National 
Lahoratory. 

graphics  and  multimedia  applications.  The  MUST- 
PAC  design  team  picked  a  platform  with  a  2'-Gbyte 
hard  drive  and  support  for  seven  SCSI-2  devices. 
The  team  then  loaded  it  with  128  Mbytes  of  RAM 
to  minimize  disk  accesses. 

While  bulkier  than  currendy  available  laptops,  the 
Indy  was  deemed  a  clear  choice  for  MUSTPAC,  as  it 
offered  video-capture  capabilities  in  a  Unix  environ¬ 
ment.  ‘We  were  considering  some  Pentium-based 
notebooks,  but  there  was  no  way  we  were  going  to 
get  the  quality  video  capture  we  needed  within  the 
time  frame  we  needed  it,”  notes  Litdefield,  “What’s 
more,  since  our  visualization  software  already  ran  in 
Unix,  we  didn’t  have  to  do  a  port  to  Windows  NT.” 

There  were  tradeoff.  One  of  the  drawbacks  to  the 
Indy  platform  was  that  it  required  a  stand-alone  key¬ 
board.  “In  theory,  we  could’ve  done  the  whole  thing 
with  a  touch  panel,”  says  litdefield.  “But  we  knew  the 
users  we  were  targeting  wanted  not  just  a  3-D  ultra¬ 
sound  system,  but  a  more  general-purpose  platform.” 
For  future-generation  MUSTPAQ,  Utdefield  indi¬ 
cates  a  Pentium  portable  with  integrated  keyboard  will 
likely  eliminate  Ae  bulk  of  the  stand-alone  keyboard. 

Since  the  Indy  doesn’t  come  with  an  integrated 
display,  the  MUSTPAC  designers  opted  to  add  an 
SGI  presenter  panel  to  the  backpack  configuration. 
“We  needed  1,024  x  1,280-pixel  resolution  with 
CRT-quality  gray  scale,”  explains  Litdefield.  “Physi¬ 
cians  performing  diagnosis  at  remote  locations  use  a 
standard  analog  CRT.” 

Image  objections 

“A  common  gripe  among  doctors  about  ultrasound- 
captured  static  images  is  you  can’t  tell  what  you’re 
looking  at  unless  somebody  tells  you  where  the 
probe  was  pointed,”  explains  Litdefield.  “A  standard 
joke  is  you  can’t  tell  the  difference  between  an  ovar¬ 
ian  cyst  and  a  gall  bladder.” 


A  problem  for  the  MUSTPAC  designers  was 
the  realization  that  trained  ultrasound  diagnosti¬ 
cians  were  used  to  working  in  a  2-D  environment, 
not  with  a  3-D  dataset.  “Through  years  of  training, 
doctors  have  developed  great  amounts  of  skill  and 
hand-eye  coordination  in  terms  of  being  able  to  in¬ 
fer  3-D  anatomy — based  not  only  on  where  they 
think  their  hand  is  pointing  right  now,  but  also 
how  an  image  changes  as  they  move  their  hand,” 
says  Litdefield. 

“Since  the  person  doing  the  diagnosis  wasn’t  go¬ 
ing  to  be  physically  located  with  the  patient,  and 
wouldn’t  have  direct  control  over  the  imaging,” 
continues  Littlefield,  “we  felt  it  was  very  important 
to  give  the  operators  a  familiar  interface  that  would 
permit  them  to  exploit  all  their  standard  skills.” 

“It’s  kind  of  counter-intuitive,”  adds  Macedonia. 
“People  who  aren’t  in  medicine  typically  think, 
‘Gosh,  it  seems  like  a  3-D  image  would  be  a  lot  more 
useful  than  a  2-D  image.’  But  the  fact  of  the  matter 
is  that  the  experienced  radiologist  uses  a  2-D  image 
and  does  3-D  reconstruction  in  his  or  her  head.” 

One  answer  to  the  visualization  problem  was  sitting 
just  down  the  hall  in  Littlefield’s  oflSce.  “One  day  I  was 
mulling  over  how  to  ^Ive  this  problem  and  I  happened 
to  walk  through  our  multimedia  lab,”  he  remembers. 
“Sitting  on  the  table  was  a;  fancy  joystick,  with  XYZ 
pitch,  roll,  and  yaw.  What  more  natural  thing  to  do  than 
to  sinply  let  the  software  listen  to  that  device  and  put  the 
corresponding  inages  up  on  the  screea>  We’ll  fake  a  real¬ 
time  exarmnation,  I  thought.” 

Out  of  that  idea  came  a  virtual  ultrasound,  or  im¬ 
mersion,  probe.  Offering  equivalent  6-D  sensing  ca¬ 
pability  (3-D  position  plus  2-p  tilt  and  rotation),  the 
probe — with  the  help  of  special  volume-visualization 
software — can  be  used  to  re-slice  the  ultrasound  in¬ 
formation  along  arbitrary  so-called  cutting  planes. 

Software  ain't  easy 

While  the  MUSTPAC-1  development  project  relied 
largely  on  off-the-shelf  components  to  minimize  de¬ 
sign  time,  software  was  another  story.  Much  of  the 
innovative  integration  work  for  the  MUSTPAC 
came  about  in  software  development. 

The  Fraunhofer  Center  for  Research  in  Com¬ 
puter  Graphics  (CRCG — ^Providence,  R.I.)  played 
a  key  role.  It’s  a  non-profit  computer  graphics  re¬ 
search  group  specializing  in  the  study  of  volume  vi¬ 
sualizations,  virtual  environments,  collaborative 
work  tools,  and  user  interfaces. 

.  Researchers  at  the  center  modified  TelelnVIVb, 
3-D  ultrasound  visualization  software  based  on  In- 
ViVo  3-D  volume-visualization  code  developed  by 
Dr.  Georgios  Sakas  at  the  institute’s  Darmstadt,  Ger¬ 
many,  facility.  Featuring  network  collaboration  tools 
for  remote  consultation  and  system  control,  Telein; 
ViVb  permits  physicians  to  view  a  range  of  data  sets, 
including  CAT  (computer- assisted  tomography), 
MRI  (magnetic  resonance  ima^g),  ^v'lRA  (magnetic 
resonance  angiography),  and  PET  (positron  emission 
tomography)  scans.  It  also  allows  users  to  exchange 
and  manipulate  the  data  sets  via  ISDN  or  ATM  net- 
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worn.  CRCGmademcdificationsoft±iesofhvare  to  gicai  Hospital  (MASH)  units,  the  pounds,  the  MUST- 

enable  the  use  of  the  virtual  ultrasound  probe.  MUSTPAC-1  was  initially  designed  to  PAC-l  tTtclud^s  an 

CRCG’s  software  used  position  and  orientation  operate  from  a  standard  120-V  ac  power  ^^^ound  systein,  a 

information  derived  by  the  probe  to  re-slice  the  3-D  line.  However,  in  the  interest  of  making 

ultrasound  data  along  arbitrary  cutting  planes.  The  the  unit  truly  useful  in  combat,  the  sys-  ASUican  (^a^Us 

screen  image  is  updated  in  real  time  so  that  diag-  tern  also  needed  a  battery-powered  dp-  workstation-class 

nosticians  see  the  3-D  MUSTPAC  scans  as  conven-  tion.  It  was  supplied  with  an  off-the-  computef^with 

,tional  2-Dimages.  shelf,  30-lb  uninterruptible  power  ^^^^^^^ompUtes 

•  ^  supply.  “The  MUSTPAC  is  sufBdently 

GUI  anxiety  low  power  that  we  could  get  45  minutes 

Concurrendy  with  hardware  and  TeleInViVo  kite-  of  continuous  operation  off  a  standard 
gration,  programmers  at  Pacific  Northwest  National  UPS  system,”  says  Littlefield. 

Laboratory  also. wrote  code  to  bring  the  system’s  Similarly,  the  system’s  communications  options 
components  together.  One  of  the  more  challenging  were  significantly  narrowed  when  initial  require- 
aspects  of  this  was  writing  video-capture  software  ments  called  for  its  use  ivith  established  Signal  Corps 
and  a  graphical  user  interface.  “Malting  it  run  ac-  connections.  “They  said,  ‘Look,  we’ll  just  give  you  a 
ceptably  fast  was  an  ongoing  exercise  in  frustration,”  lOBase-T  connection  talking  TCP/IP.’  We  said, 
admits  Littlefield.  ‘Great,’”  retells  Littlefield.  But  down  the  road,  lit- 

For  the  video  input  side,  MUSTPAC  designers  defield’s  designers  still  see  an  opportunity— particu- 
built  their  applications  on  top  of  VL,  a  library  from  larly  if  the  system  is  brought  to  commercialization. 

SGI.  But  since  VL  didn’t  handle  any  functions  on  They  want  to  add  BF  wireless  capability  in  future  it- 
the  output  side,  the  team’s  programmers  had  to  erations  of  the  design.  Dtdefield  sees  KF  satellite  . 

.  write  custom  code  to  glue  the  VL-based  applica-  links,  such  as  those  based  on  the  Immarsat  birds,  as 
tions  with  the  user  interface.  The  latter  was  written  the  most  likely  scenario  for  this.  Dual-mode  cell- 
in  TCL,  a  language  and  support  library  that  comes  phone  technology  is  also  feasible,  he  says, 
out  of  the  X  Windows  world. 


While  the  actual  amount  of  code  proved  relatively 
small,  the  task  of  managing  the  video  input,  as  well  as 
handing  off  .the  imagery  to  TCL  for  display  on  the 
screen  and  as  a  backdrop  for  the  user  interface, 
proved  to  be  a  very  difficult  task.  “TCL  is  designed  to 
use  an  event  loop  philosophy;  it  provides  an  event 
handler  of  its  own,”  Littlefield  explains.  “SGI’s  VL  is 
also  designed  to  use  the  event-loop  philosophy,  but  it 
uses  an  event  loop  of  its  own.  Getting  those  two 
glued  together  so  that  all  the  events  on  both  video  in¬ 
put  and  X  Windows  sides  were  properly  interleaved 
turned  into  a  matter  of  several  weeks  of  hair  tearing.” 


Tuzla-tested 

The  MUSTPAC-1  prototype  was  shipped  in  July  of 
1996,  but  its  first  real-world  trial  came  a  mon^  later 
when  Macedonia  took  the  system  to  the  212th 
MASH  at  Camp  Bedrock  in  Tuzla,  Bosnia.  Con- 
•  cerns  about  reliability  were  quickly  put  to  rest  when 
Macedonia  went  to  unload  the  system  after  it  took  a 
truck  ride  down  a  seven  mile  road  called  Crater  Al¬ 
ley.  To  his  dismay,  Macedonia  found  the  system  had 
bounced  off  its  palette  and  was  lying  in  a  comer  of 
the  back  of  the  truck.  “It  had  the  crap  kicked  out  of 
it,”  he  unabashedly  declares. 


No  batteries  included 

Power-supply  issues  were  also  critical.  At  first,  oper¬ 
ating  power  was  not  a  concern  for  the  design  team. 
Intended  for  use  in  armed  forces  Mobile  Army  Sur- 


But  xhc  imt  didn’t  fail.  Once  set  up  at  A^IASH 
headquarters,  the  system  was  fired  up  and  linked  to 
the  Army’s  local  lObase-T  Ethernet  network.  Data 
was  transmitted  back  through  a  microwave  link  to  a 
base  miles  away  and  then  fed  through  a  Ku-band- 
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STRA  T  EG  IES 


DEVEL  O  PMENT 


satellite  link  to  another 
MUSTPAC  unit  in  Land- 
stuhl,  Germany.  That  field 
test  proved  clearly  that  data 
could  be  collected  and  trans¬ 
mitted  by  personnel  inexpe¬ 
rienced  with  ultrasound 


Users  with  little 
ultrasound  training 
can  collect  data  for 
remote  analysis  using 
theMXJSTPAC-l 

backpack  telemedicine 
system. 


technology. 

‘T  initiiUiy  thought  I  was  going  have  to  do  all  the 
scans,”  recalls  Macedonia.  ‘"But  when  I  got  there, 
everyone  asked  to  try  it.  Eventually,  we  had  a  medic, 
a  Russian  nurse,  and  even  a  chaplain  try.it  out.”  The 
*  ability  of  diagnosticians  to  use  the  virtual  ultrasound 
probe  also  exceeded  expectations.  “It  became  very 
clear  it  didn’t  matter  who  was  doing  a  scan,”  re¬ 


counts  Macedonia. 


Some  of  the  results  of  the  trip  were  completely 
unexpected.  “We  went  into  Bosnia  thinking  we  were 
taking  in  a  3-D  ultrasound  machine  and  came  out  of 
the  experience  realizing  we  had  what  could  be  called 
a  portable  telemedicine  system — that  at  its  core  had 
ultrasound,”  avows  Macedonia. 

Since  the  system  does  fi'ame  capture,  users  at  the 
jMASH  quickly  began  experimenting  with  CCD  cam¬ 
eras  and  other  instruments  to  capture  eye  exams,  ear 
exams,  and  even  dental  exams.  They  were  anxious  to 
send  them  in  a  store-and-forward  feshion  back  to  the 


rear.  “The  people  at  the  MASH  tau^t  us  that  it’s  re¬ 


ally  a  jportable  telemedicine  system  that  allows  you  to 
examine  a  patient  firom  the  top  of  their  head  to  bottorn 
of  their  feet,”  jocularly  notes  Macedonia. 


A  leading  designer  and  manufacturer  of  keyboards  and 
keypads  for  portable  input  products,  our  technologies  include: 


'  Flat  Pane!  /  Membrane 
•  Full-Trayel  /  Notebook 
'  Hand-Held  Products 
’  Complete  Assemblies 


•Mid-Travel /Elastomer 

•  Full  Design  Services 

•  Models  &  Prototypes 

•  Complex  Control  Panels 


Call  (800)444-5923 

ADVANCED  INPUT  DEVICES 
W  A  WM  West  250  AID  Drive 

Coeur  d'Alene.  ID  8381 4 


ADVANCED  INPUT  DEVICES 
West  250  AID  Drive 
Coeur  d'Alene,  ID  83814 
Telephone:  208-765-8000 
Fax:  208-772-7613 
httpV/www.advanced-input.com 
E-mail:  sales@advanced-input.com 


A  lighter  future 

At  the  moment,  the  MUSTPAC  design  team  is  fo¬ 
cusing  on  reducing  system  weight  and  cost;  a.riew- 
generation  system  is  in  the  works.  Key  to  achieving 
the  goal  will  be  a  migration  to  a  lighter  and  more 
compact  Pentium-based  platform  that  will  include 
keyboard,  display,  and  processor  in  a  single  unit. 
“That  should  eliminate  about  10  pounds,”  believes 
Littlefield.  “It’ll  eliminate  a  bunch  of  cable  and  sep¬ 
arable  interconnect  problems  as  well.” 

“The  development’s  a  continuous  evolution,”  con- . 
eludes  Macedonia.  “I  don’t  see  an  end  to  the  MUST¬ 
PAC- 1  project  until  we  create  a  handheld — ^not  a 
backpack — that  allows  me  to  have  a  look  at  people’s 
bodies  and  figure  out  what’s  going  on  inside.”  3 


I  For  more  information.. 
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“Backpack  remote  medicine  proves  its  worth”,  Janes 
International  Defense  Review,  February  1997 


Backpack  remote  medicine  proves  its  worth 


A  portable  telemedicine  system  provid¬ 
ing  ultrasound  facilities  for  use  in 
field  hospitals  has  undergone  suc¬ 
cessful  trials  vvith  the  US  Army  in  Bosnia. 
Pacific  Northwest  Laboratory  (PNL)  devel¬ 
oped  the  Medical  UltraSound,  Three- 
dimensional  and  Portable  with  Advanced 
Communication  (MUSTPAC)  system 
under  a  rapid-prototyping  contract  firom  the 
US  Defense  Advanced  Research  Projects 
•  Agency.  The  design  exploits  existing  technol¬ 
ogy  to  make  referral  and  diagnosis  easier  in 
the  field,  leading  to  improved  patient  care 
and  a  reduction  in  the  number  of  unneces¬ 
sary  evacuations. 

PNL  says  that  MUSTPAC  is  unique  in 
that  it  is  effective,  requires  very  litde  training, 
does  not  need  a  highly  skilled  operator  at  the 
patient’s  side,  and  operates  well  even  in  a 
low-bandwidth  store-and-forward  file- 
transfer  mode.  The  laboratory  is  developing 
a  production-standard  version  that  is  lighter, 
more  compact,  and  uses  different  computer 
hardware. 

MUSTPAC  consists  of  a  field  unit 
installed  in  a  backpack,  allowing  it  to  be 
operated  as  far  forward  as  battalion  aid  sta¬ 
tions,  together  with  a  separate  virtual  ultra¬ 
sound  probe  and  high-resolution  color 
monitor.  In  its  prototype  form,  the  backpack 
contains  a  ‘three-dimensional  [3-D]  paddle’ 
electromechanical  scanner;  Silicon  Graphics 
Presenter  flat-panel  display;  Hitachi  EUB- 
905  ultrasound  machine;  Silicon  Graphics 
Indy  computer;  teleconferencing  video  cam¬ 
era;  and  keyboard  with  integral  touchpad. 
The  separate  Immersion  Probe  —  a  virtual 
ultrasound  probe  —  provides  expert  diag¬ 
nosticians  with  a  familiar  interface  that  is 
easy  to  learn  and  to  use. 

The  system  permits  on-the-spot  visualiza¬ 
tion  of  internal  bleeding,  damage  to  solid 
organs,  and  penetrating  injuries.  It  operates 

The  MUSTPAG-1  prototype 
consists  of  a  virtual 
ultrasound  probe  (extreme 
left  of  photograph),  high- 
resofution  color  monitor 
(next  to  the  probe),  and 
a  backpack  (center) 
containing  the  other 
elements.  These  comprise 
an  electromechanical 
scanner  (extreme  right), 
flat-panel  display  (next 
to  it),  ultrasound  machine 
and  its  associated 
computer  (both  in 
backpack), 

teleconferencing  camera 
(on  backpack),  and 
keyboard  with  integral 
touchpad  (foreground). 


by  scanning  a  fairly  large  volume  of  the 
patient  at  one  time,  so  that  diagnosis  during 
the  scan  is  not  required.  The  operator  places 
the  scanner  on  the  patient’s  abdomen  for 
about  10s,  while  the  system  acquires  a  large 
volume  of  3-D  data.  This  requires  minimal 
training.  US  Marine  Corps  Capt  (now  Maj) 
Christian  Macedonia,  the  MUSTPAC  Prin¬ 
cipal  Medical  Investigator,  talked  a  Russian 
nurse  through  the  process,  via  an  interpreter, 
in  about  5min  during  trials  in  Bosnia. 

The  equipment  also  provides  a  remote 
diagnostician  with  a  fanfiliar  interface.  The 
‘virtual  ultrasound  probe’  consists  of  a  modi¬ 
fied  Immersion  Probe  or  an  equivalent  6-D 
sensing  capability  (3-D  position,  2-D  tilt, 
and  rotation).  TeleInViVo  visualization  soft¬ 
ware  developed  by  the  Fraunhofer  Center 
for  Research  in  Computer  Graphics  uses 
probe  position  and  orientation  inputs  to 


Consulting  physicisns  on  onothor  continent  con  use 
MUSTPAC's  virtual-reality  technology  to  obtain  2-D 
diagnostic  views  from  3-D  ultrasound  data.  An 


Immersion  Probe  provides  high-resolution  pictures 
of  scans  taken  elsewhere. 


‘reslice’  the  3-D  ultrasound  data  along  arbi¬ 
trary  cutting  planes.  The  screen  view  is 
updated  in  real  time  (5-10  times  a  second), 
so  that  diagnosticians  can  work  with 
MUSTPAC  3-D  scans  in  the  same  way  that 
they  would  with  a  real  patient  and  a  conven¬ 
tional  2-D  ultrasound  system. 

PNL  delivered  the  MUSTPAC- 1  proto¬ 
type  to  the  US  Army  Medical  Research  and 
Materiel  Command’s  Medical  Advanced 
Technology  Management  Office  in  July  last 
year.  Following  a  series  of  pre-deployment 
evaluations  supervised  by  the  Center  for 
Total  Access  at  Fort  Gordon,  it  was  shipped 
to  Landstuhl  in  Germany.  In  early  August 
the  equipment  began  a  month-long  deploy¬ 
ment  with  the  212th  Mobile  Army  Surgical 
Hospital  at  Camp  Bedrock  in  Tuzia,  Bosnia. 

I  It  transmitted  data  through  a  microwave  link 
to  Eagle  Base,  12km  away,  and  then  fed 
through  a  Ku-band  satellite  link  to  another 
MUSTPAC  unit  in  Landstuhl.  Fotir  other 
receiving  sites  were  situated  in  the  US, 

The  MUSTPAC- 1  prototype  weighs  just 
under  40kg.  The  production-standard  ver¬ 
sion  now  under  development  will  be  more 
compact,  weigh  about  27kg,  and  incorporate 
a  laptop  computer  in  place  of  the  Indy  unit. 
The  equipment  may  then  enter  limited 
production  under  a  government/industry 
parmership.  Potential  users  include  rapid- 
reaction  forces  and  naval  units. 

In  the  latter  case,  MUSTPAC  could  assist 
in  detecting  gallstones,  diagnosing  kidney 
disease,  and  meeting  the  need  for  gynecolog¬ 
ical  care  that  has  resulted  firom  the  deploy¬ 
ment  of  women  aboard  warships.  The 
system  could  also  form  an  adjunct  to  image- 
directed  surgery  for  immediate  treatment. 
MUSTPAC  coiild  show  the  position  of  both 
injuries  and  surgical  instruments,  supporting 
tasks  such  as  the  injection  of  glue  to  stitch 
internal  wounds.  MH 
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Appendix  6: 

Presentation  to  MUSTPAC-2  Expert  Review  Panel, 
Augusts,  1997 


MUSTPAC  Objective 

Develop  and  make  available  for  routine  use 
an  effective  ultrasound  telemedicine  system 
that  does  not  require  diagnostic  skills 
at  the  patient’s  location. 


Design  Philosophy 

♦  Evolutionary  development 

♦  Alternating  advances  In  hardware  and  software 


MUSTPAC-1  (4Q1996) 

♦  Investigational  device,  IRB  approval 

♦  85-pound  backpack 

♦  Data  acquisition  via  motor-driven  linear  device 

{1st  generation) 

♦  B-modeonly 

♦  Manufacturing  and  support  capability:  none  in  place 

♦  Rapid  prototype  assembly  incorporating: 

-  small  desktop  computer  (SGI  Indy) 

-  separate  flat  panel  display  and  keyboard 

-  separately  packaged  ultrasound  system  (Hitachi  EUB-905) 


-  user  interface  split  between  computer  and  ultrasound  system 

♦  irix  (SGI  Unix)  operating  system  plus  Windows  applications 

♦  TCP/IP  communications,  no  encryption 


MUSTPAC-2  (4Q1998) 

♦  FDA  51 0(k)  approved 

♦  65-pound  wheeled  case 

♦  Data  acquisition  via  two  mechanisms 

-  primary:  motor-driven  linear  scan  device  (2nd  generation) 

-  secondary:  freehand  scanning  (via  magnetic  position  sensor) 

♦  B-mode  only 

♦  Manufacturing  and  support  capability:  target  50  units,  $1  OOK/unit 

♦  Modest  level  of  integration  incorporating: 

-  laptop  style  computer  subsystem 

-  separately  packaged  ultrasound  system  (Hitachi  EUB-905) 

-  user  interface  split  between  computer  and  ultrasound  system 

♦  Windows/NT  operating  system 

♦  TCP/IP  communications,  encryption  provided 


MUSTPAC-2  (Interim  -  4Q1997) 

Investigational  device,  initial  510(k)  application  submitted 

♦  65-pound  wheeled  case 

Data  acquisition  via  one  mechanism 

-  primary:  motor-driven  linear  scan  device  (2nd  generation) 

♦  B-mode  only 

-►  Manufacturing  and  support  capability:  developing 

♦  Modest  ievel  of  integration  incorporating: 

-  laptop  style  computer  subsystem 

-  separately  packaged  ultrasound  system  (Hitachi  EUB-905) 

-  user  interface  split  between  computer  and  ultrasound  system 
■^  Solaris  operating  system  plus  Windows  applications 

♦  TCP/IP  communications,  encryption  provided 


MUSTPAC-3  (4Q  2000) 

25-pound  briefcase 
Multiple  ultrasound  modes; 

-  B-mode 

-  Doppler  power  angio 

-  motion  capture  and  display 

Data  acquisition  via  three  mechanisms 

-  primary:  freehand  scanning  (magnetic  free  space  sensor) 

-  optional:  3-D  paddle  (motor-driven  linear  scan  device) 

-  optional:  motion  capture  via  slow  linear  scan 
Designed  for  civilian  as  well  as  military  use 

Manufacturing  and  support  capability:  target  500  units,  $30K/unit 
Highly  integrated  system  incorporating 

-  board-level  ultrasound  subsystem 

-  low  cost  OEM-type  computer  subsystem 


Clinical  Study 


♦  Georgetown  University  Medical  Center 

♦  Title:  Reformatting  of  three-dimensional 
ultrasound  data  sets  for  remote  consultation: 
Are  fetal  anomalies  detectable  and  are 
biometric  measurements  accurate  in  3D 
Ultrasound  data  sets 

♦  60  patients:  20  known  anomalies,  40  normal 

♦  2-D  and  3-D  scans  archived,  blind  comparison 
of  interpretation  and  biometric  measurements 


Independent  Publications 

♦  Jane’s  International  Defense  Review;  Feb. 1997 

♦  Portable  Design  Magazine,  June  1997  (cover  article) 

Honors 

♦  Discover  Award  Winner,  June  1997  (Discover  Magazine) 
Computer  Hardware  and  Electronics 

♦  Solution  of  the  Year,  June  1997  (Advanced  Imaging 
Magazine),  Scientific  Visualization 


■■■■■■■ 


r 


Press  Coverage 

♦  Associated  Press  -  numerous  articles 

♦  Broadcast  TV  (CBS,  CNBC,  Discovery  Channel) 

Invited  Presentations 

♦  AUSA  annual  meeting 

♦  AUSA  Telemedicine  Conference 

♦  Military  Medical  Capabilities  Conference 

♦  Society  of  Minimally  Invasive  Theraputics  (Japan) 

♦  Spanish  Society  of  OB/GYN  (Spain) 

♦  Tribal  Healthcare  2000  Conference 

♦  AMEDD  Conference  on  Force  Structure  and 
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Appendix  7: 

MUSTPAC-2  Prototype,  10/15/97 


MUSTP AC-2  Prototype,  10/15/97 


http://aims.pnl.gov:2080/MUSTPAC-2/ 


MUSTP  AC-2  Prototype,  10/15/97 

Background 

MUSTPAC  (Medical  UltraSound,  Three-dimensional  and  Portable,  with  Advanced  Communications)  is 
a  telemedicine  system  that  uses  3-D  ultrasound  data  acquisition.  It  provides  the  unique  capability  of 
allowing  an  effective  ultrasound  examination  to  be  performed  without  requiring  ultrasound  diagnostic 
skills  at  the  patient's  location.  The  MUSTP  AC-2  system  is  a  successor  to  the  highly  regarded 
MUSTPAC- 1  and  operates  on  similar  principles.  System  design  and  field  experiences  of  the 
MUSTPAC- 1  are  described  in  an  accompanying  report,  "MUSTPAC- 1 :  3-D  Ultrasound  Telemedicine 
Tool  for  Deployment  Situations  in  Bosnia  and  the  European  Theater".  An  earlier  description  of  the 
MUSTP  AC-1  ■  dated  12/96.  is  also  available 

Following  are  pictures  of  the  MUSTP  AC-2  prototype  system,  taken  during  a  system  integration  test  on 
October  15,  1997. 
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SolidWorks  CAD  drawing  of  module  and  case  layout. 
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External  appearance  of  cases  (1"  grid) 
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10/15/97 
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MUSTPAC-2  Prototype,  10/15/97 
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MUSTPAC-2  Prototype,  10/15/97 


Closeup  of  3-D  Paddle  Scanner 
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Appendix  8: 

Computational  Focusing  -  Support  for  Off-Axis  Spherical 
Illumination  with  2-D  Array  Sensors 
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Oblique  Plane-wave  based  Reconstruction 
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Plane-wave  Illumination  Spherical-wave  Illumination 


